Program Search with a Hierarchical Variable Lenght Representation: Genetic Programming, Simulated Annealing and Hill Climbing

This paper presents a comparison of Genetic Programming(GP) with Simulated Annealing (SA) and Stochastic Iterated Rill Climbing (SIRC) based on a suite of program discovery problems which have been previously tackled only with GP. All three search algorithms employ the hierarchical variable length representation for programs brought into recent prominence with the GP paradigm [K-92]. We experiment with three GP crossover operators and a new hierarchical variable length mutation operator developed for use in SA and SIRC. The paper provides a comparison among GP, SA and SIRC in terms of likelihood of success, required evaluations and program characteristics. This is the first reported experimentation with the goal of program discovery using SA and SIRC. We feel it is not intuitively obvious that mutation-based adaptive search can handle program discovery yet, to date, for each GP problem we have tried, SA or SIRC also work. One important conclusion from the experimentation is to recognize the general value_ of a hierarchical variable length representation for program induction as it is distinct from different search strategies and operators complementary to both the strategy and representation. Based upon comparable results among GP, SA and SIRC, the results emphasize that a search strategy should be chosen according to the characteristics of the landscape (which is determined by fitness function and representation) and with regard to factors such as algorithmic simplicity and computational complexity. Introduction Genetic Programming (GP) [K-92] is a recent paradigm in the lineage of search techniques for performing program induction [F-58, FDN-59, L-83, S-83, C-85, FD-87]. The sources of its effectiveness are: I) an evolution inspired exploitation and exploration mechanism and, 2) the use of a hierarchical, variable length representation for programs. In the former respect, GP uses fitness proportional selection (a la natural selection) as a basis for choosing which individuals among a population of programs will be the parents of the next generation and a genetics based crossover operation for transforming two parents into two novel offspring. This provides a robust heuristic search approach [H-92, G-89, K-92]. Regarding the latter source, the hierarchical representation is supported by a genetic crossover operator which ensures the syntactic closure of all programs and permits programs of different lengths to be generated when two "parent" parents are "crossed over". Thus in GP a fixed size and structure of programs in the search space does not have to be a priori specified. Koza attributes one of GP's strengths to this flexibility [K-92, Ch 4]. One of the goals of this paper is to isolate GP's representation scheme from its evolution-based adaptation to understand the degree to which each is crucial in solving a program discovery problem. Given that a hierarchical variable length representation is used, we would like to obtain a better notion of when GP is the most appropriate paradigm for solving a particular program , [Revision date: 6/6/941