Complexity of Functions with Cartesian GP and Recursion

Complexity of Functions with Cartesian GP and Recursion. John Woodward. School of Computer Science, The University of Birmingham, United Kingdom. 1 OVERVIEW Genetic Programming (GP) often uses a tree form of a graph to represent solutions. An extension to this representation, Automatically Defined Functions (ADFs) is to allow the ability to express modules. We proved that the complexity of a function is independent of the primitive set (function set and terminal set) if the representation has the ability to express modules. This is essentially due to the fact that if a representation can express modules, then it can effectively define its own primitives at a constant cost. Cartesian Genetic Programming (CGP) is a relative new type of representation used in Evolutionary Computation (EC), and differs from the tree based representation in that outputs from previous computations can be reused. This is achieved by representing programs as directed acyclic graphs (DAGs), rather than as trees. Thus computations from sub trees can be reused to reduce the complexity of a function. We prove an analogous result; the complexity of a function using a Cartesian Program (CP) representation is independent of the terminal set (up to an additive constant), provided the different terminal sets can both be simulated. This is essentially due to the fact that if a representation can express Automatic Reused Outputs, then it can effectively define its own terminal set at a constant cost. 5 Cartesian Genetic Programming 2 Definitions The size, s(p), of a program is the number of nodes it contains. A primitive set, p 1, is the union of the function set and terminal set. A dictionary Dp 1 p 2 is the set of programs which express each of the primitives in set p 2 in terms of programs written in p 1. The complexity of a function C(f) p 1 is the size of the smallest program which expresses that function (w. r. t. a given primitive set). The complexity of a dictionary C(Dp 1 p 2) is the size of the smallest dictionary. The translation of primitive sets is the process of taking a function expressed using one primitive set and expressing it using another. Two primitive sets are equally expressive if they can express the same class of functions in finite size. Genotype 2 1 3, 3 2 1, 2 3 2, 6 6 3, 4 2 4, 2 6 3, … Interpret as (input, function) the first pair being inputs, the last number Representing a function (+, -, *, /) interpreted as (1, 2, 3, 4). This representation contains neutrality. The program has 3 inputs (1, 2, 3) and 3 outputs (10, 11, 12). The outputs of each node are labelled consecutively from 4. Information flows from left to right, there is reuse of computed values. 1 2 6 * 4 10 * 7 2 2 6 3 7 4 + 5 + 11 / 8 2 2 7 2 6 * 12 * 9 3 9 6 6 Another view of CGP 3 Automatically Defined Functions ADFs allow new functions to be defined. Once defined, the function can be called as many times as needed. Essentially, a new primitive set can be defined in terms of the current primitive set. The main tree on the left calls the adf on the right twice. An arbitrary sub tree can be extracted and expressed as an ADF. + - adf 1 4. 1 adf 1 c If we view CGP as a tree, reuse can be expressed as overlapping sub trees (All the way down to the root nodes – see shaded circle). As the type of reuse is constrained, the central result only holds for translation of terminals and not the whole primitive set. a 1. 0 + arg 1 + a b 7 Visualization of CGP Proof arg 2 4 Translation of primitive set for ADFs Given a program expressed in terms of p 1 (the single box), We can construct a program in terms of p 2 in two stages. First, construct the set of ADFs which express p 1 in terms of p 2 (i. e. the dictionary D). Second, in the original GP program, remove references to p 1 and Refer instead to the appropriate ADF corresponding to that primitive in the dictionary. C(f)p 1 GP expressed in terms of p 1 C(f)p 2 GP expressed in terms of p 1 Given two primitive sets p 1 (= F U T 1) and p 2 (= F U T 2) i. e. they contain the same function set but different terminal sets. We assume they are equally expressive. Given a function expressed in terms of p 1 (the box on the right), We can construct a program expressed in terms of p 2, By constructing a program which converts T 2 into T 1 (the box on the left) and feeding its outputs into the original program. Cartesian Program. T 1 expressed in terms of F and T 2 Function expressed in terms of T 1 and F. 8 Summary Set of primitives p 1 expressed as ADFs in terms of p 2 The complexity of a function when expressed in one primitive set, Is sandwiched between the complexities then expressed using a different primitive set. Note that the constants are different, but related. In general we can say these bounds are the tightest obtainable. This result was first obtained for ADFs, but is found to include recursion and iteration. The result for CGP only holds for translation of terminal sets as the degree of reuse in CGP is limited.