On the Success Rate of Crossover Operators for Genetic Programming with Offspring Selection

Genetic programming is a powerful heuristic search technique that is used for a number of real world applications to solve amongst others regression, classification, and time-series forecasting problems. A lot of progress towards a theoretic description of genetic programming in form of schema theorems has been made, but the internal dynamics and success factors of genetic programming are still not fully understood. In particular, the effects of different crossover operators in combination with offspring selection are largely unknown. This contribution sheds light on the ability of well-known GP crossover operators to create better offspring when applied to benchmark problems. We conclude that standard (sub-tree swapping) crossover is a good default choice in combination with offspring selection, and that GP with offspring selection and random selection of crossover operators can improve the performance of the algorithm in terms of best solution quality when no solution size constraints are applied. 1 Genetic Programming Genetic programming (GP) is a generalization of genetic algorithms first studied at length by John Koza [5]. Whereas the goal of genetic algorithms is to find a fixed length vector of symbols that encodes a solution to the problem, the goal of genetic programming is to find a variable-length program that solves the original problem when executed. Common practice is to use a tree-based representation of computer programs similar to so called symbolic expressions of functional programming languages such as LISP. Genetic programming is a powerful heuristic search method that has been used successfully to solve real world problems from various application domains, including classification, regression, and forecasting of time-series [9, 16]. The work described in this paper was done within HEUREKA!, the Josef Ressel center for heuristic optimization sponsored by the Austrian Research Promotion Agency (FFG). Offspring selection [1] is a generic selection concept for evolutionary algorithms that aims to reduce the effect of premature convergence often observed with traditional selection operators by preservation of important alleles [2]. The main difference to the usual definition of evolutionary algorithms is that after parent selection, recombination and optional mutation, offspring selection filters the newly generated solutions. Only solutions that have a better quality than their best parent are added to the next generation of the population. In this aspect offspring selection is similar to non-destructive crossover [21], soft brood selection [3], and hill-climbing crossover [13]. Non-destructive crossover compares the quality of one child to the quality of the parent and adds the better one to the next generation, whereas offspring selection generates new children until a successful offspring is found. Soft brood selection generates n offspring and uses tournament selection to determine the individual that is added to the next generation, but in comparison to offspring selection the children do not compete against the parents. Hill-climbing crossover generates new offspring from the parents as long as better solutions can be found. The best solution found by this hill-climbing scheme is added to the next generation. The recently described hereditary selection concept [11, 12] also uses a similar offspring selection scheme in combination with parent selection that is biased to select solutions with few common ancestors.