John R Koza Edited by J Wiebe 1

John R. Koza [Edited by J. Wiebe] 1

GENETIC PROGRAMMING 2

Notes • [added by J. Wiebe] • A Field Guide to Genetic Programming, 2008, Poli, Langdon, Mc. Phee, Koza (easy to find via Google) • Author of these slides, John Koza, is a pioneer in the field 3

THE CHALLENGE "How can computers learn to solve problems without being explicitly programmed? In other words, how can computers be made to do what is needed to be done, without being told exactly how to do it? " Attributed to Arthur Samuel (1959) 4

CRITERION FOR SUCCESS "The aim [is]. . . to get machines to exhibit behavior, which if done by humans, would be assumed to involve the use of intelligence. “ Arthur Samuel (1983) 5

REPRESENTATIONS • Decision trees • If-then production rules • Horn clauses • Neural nets • Bayesian networks • Frames • Propositional logic • Binary decision diagrams • Formal grammars • Coefficients for polynomials • Reinforcement learning tables • Conceptual clusters • Classifier systems 6

GENETIC PROGRAMMING (GP) • GP applies the approach of the genetic algorithm to the space of possible computer programs • Computer programs are the lingua franca for expressing the solutions to a wide variety of problems • A wide variety of seemingly different problems from many different fields can be reformulated as a search for a computer program to solve the problem. 7

GP FLOWCHART 8

A COMPUTER PROGRAM IN C int foo (int time) { int temp 1, temp 2; if (time > 10) temp 1 = 3; else temp 1 = 4; temp 2 = temp 1 + 2; return (temp 2); } 9

PROGRAM TREE (+ 1 2 (IF (> TIME 10) 3 4)) 10

CREATING RANDOM PROGRAMS 11

CREATING RANDOM PROGRAMS • Available functions F = {+, -, *, %, IFLTE} • Available terminals T = {X, Y, Random-Constants} • The random programs are: – Of different sizes and shapes – Syntactically valid – Executable 12

GP GENETIC OPERATIONS • • Reproduction Mutation Crossover Architecture-altering operations 13

MUTATION OPERATION • • Select 1 parent probabilistically based on fitness Pick point from 1 to NUMBER-OF-POINTS Delete subtree at the picked point Grow new subtree at the mutation point in same way as generated trees for initial random population (generation 0) • The result is a syntactically valid executable program • Put the offspring into the next generation of the population • [Example: in class] 14

CROSSOVER OPERATION • Select 2 parents probabilistically based on fitness • Randomly pick a number from 1 to NUMBER-OFPOINTS for 1 st parent • Independently randomly pick a number for 2 nd parent • The result is a syntactically valid executable program • Put the offspring into the next generation of the population • Identify the subtrees rooted at the two picked points • [Example in class] 15

REPRODUCTION OPERATION • Select parent probabilistically based on fitness • Copy it (unchanged) into the next generation of the population 16

[Initialization] • Maximum initial depth of tree Dmax is set • Full method (each branch has depth = Dmax): – nodes at depth d < Dmax randomly chosen from function set F – nodes at depth d = Dmax randomly chosen from terminal set T • Grow method (each branch has depth Dmax): – nodes at depth d < Dmax randomly chosen from F T – nodes at depth d = Dmax randomly chosen from T • Common GP initialisation: ramped half-and-half, where grow & full method each deliver half of initial population – Ramped: use a range of depth limits

[Pseudocode for program generation method is either ‘full’ or ‘grow’] Gen(max_d, method) 1. If max_d = 0 or (method = grow and rand[0, 1] < |term_set| / (|term_set|+|func_set|)) then 1. Expr = random(term_set) 2. Else 1. Func = random(func_set) 2. For i = 1 to arity(func): 1. Arg_i = Gen(max_d – 1, method) 3. Expr = (Func, arg_1, arg_2, …) 3. Return Expr 18

Bloat • Bloat = “survival of the fattest”, i. e. , the tree sizes in the population are increasing over time • Ongoing research and debate about the reasons • Needs countermeasures, e. g. – Prohibiting variation operators that would deliver “too big” children – Parsimony pressure: penalty for being oversized • [This will come up again later]

FIVE MAJOR PREPARATORY STEPS FOR GP • • • Determining the set of terminals Determining the set of functions Determining the fitness measure Determining the parameters for the run Determining the method for designating a result and the criterion for terminating a run 20

[Issues with function sets] • Typically, Closure is required 1. Type consistency – any subtree may be used in any argument position for every function • Why? Initial tree generation, subtree generation in mutation, and crossover may generate any combination. • Require that all functions argument and return types are the same • Seems limiting, but can often be gotten around – Subcase: allowed type conversions, such as boolean to int – Subcase: make function general; some uses will ignore things • Alternative: crossover and mutation constrained to produce only type compatible programs (Section 6. 2 in the Field Guide) 21

[Issues with function sets] • Typically, Closure is required – 2. Evaluation safety • • E. g. protected values of numeric functions. Instead of throwing an exception, return a default value. E. g. , 4/0 returns 1. E. g. no-ops in planning, such as move-forward when the robot is face forward against the wall 22

[Issues with Function Sets] • Type consistency and evaluation safety may go hand in hand • Suppose type T covers all the types we want to use. • Suppose a function’s arguments should only range over a subset of values covered by T • A protected version of the function returns a default value for arguments of types the function is not actually defined over. 23

[Issues with Function Sets] • Alternative to protected functions: trap runtime exceptions and strongly reduce the fitness of programs that generate such errors • But, this may introduce many “nonsense” individuals in the population, all with similar fitness. The GP system may not be able to “find” the valid individuals 24

Structures other than Programs • In design problems, the solution may be an artifact. Bridge, circuit, etc. Functions may build structures, rather than be computer code. (We may return to this later. Before that, we’ll assume solutions are computer code. ) 25

[Fitness function] • E. g. , error between output and the desired output; payoff, in a game-playing setting; compliance of a structure with design criteria • The fact that individuals are computer programs brings up a couple issues for evaluating fitness … 26

[Fitness function evaluation] • Not simply a function application, F(X) • X is a program – X needs to be executed • On multiple inputs – So, part of specifying the fitness evaluation is specifying which inputs • Computationally expensive: multiple executions of each member of the population – Compilation? Depending on the primitive set (the terminal and function sets), the overhead of building/testing a compiler might not be worth it. So, often, evaluation is via interpreter, even though more expensive 27

[Interpreter for a expr in prefix notation, represented as a list] 1. If expr is a list then 1. Proc = expr(1) 2. Val = proc(eval(expr(2)), eval(expr(3)), …) 2. Else 1. If expr is a variable or constant then 1. Val = expr 2. Else 1. Val = expr() {terminal 0 -arity function: execute) 3. Return Val • Example in class 28