Chapter 14 Genetic Algorithms Comp 3710 Artificial Intelligence

Chapter 14. Genetic Algorithms Comp 3710 Artificial Intelligence Computing Science Thompson Rivers University

Course Outline n n Part I – Introduction to Artificial Intelligence Part II – Classical Artificial Intelligence, and Searching n n Knowledge Representation Searching n n Knowledge Represenation and Automated Reasoning n n n Search Methodologies Advanced Search Genetic Algorithms (relatively new study area) Propositinoal and Predicate Logic Inference and Resolution for Problem Solving Rules and Expert Systems Part III – Machine Learning Part IV – Advanced Topics TRU-COMP 3710 Genetic Algorithms 2

Chapter Objectives TRU-COMP 3710 Genetic Algorithms 3

Chapter Outline 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. Introduction – Why do we use Genetic Algorithms? Representations – How to represent a problem The Algorithm Fitness – How to evaluate if a chromosome is good or bad Crossover – How to generate the next generation Mutation – How to escape local maxima Termination Criteria – When do we have to stop the algorithm? Examples – What can we learn from experiences? Why genetic algorithms work? Non-standard GAs Messy genetic algorithms Predators and coevolution Project ideas TRU-COMP 3710 Genetic Algorithms 4

Reference n n Artificial Intelligence Illuminated, Ben Coppin, Jones and Bartlett Illuminated Series Introduction to Evolutionary Computing Genetic Algorithms, A. E. Eiben and J. E. Smith Java Genetic Programming Package n http: //jgap. sourceforge. net/ GA Playground n http: //www. aridolan. com/ga/gaa. html n n Very interesting Demo n n Traveling salesman problem n https: //github. com/parano/Genetic. Algorithm-TSP n http: //parano. github. io/Genetic. Algorithm-TSP/ Genetic programming Maze Marchers – http: //www. cs. northwestern. edu/~fjs 750/netlogo/final/mazemarchers. html TRU-COMP 3710 Genetic Algorithms 5

1. Introduction TRU-COMP 3710 Genetic Algorithms 6

Natural Evolution n Darwin : The Origin of Species n Natural evolution has produced many optimal forms n n n Important principles used by natural evolution n n sharks, ichthyosaurs, and dolphins – very good at swimming albatross wings – very good at soaring Assortment (diversity): recombination of genetic material Selection (cumulative improvement): survival of the fittest Random mutation (random walk) In this class, I am not taking any side of Evolutionism and Creationism. TRU-COMP 3710 Genetic Algorithms 7

Word. Evol: Cumulative Selection Cumulative selection with simple random selection: randomly chooses a character for the first letter in the string, then randomly choose a character for second letter, and so on. COMPUTER D CR K P OMPUTER TXQ A S ZD IPKR BFY n n [Q] What do you think of this idea? [Q] When we consider one trial as one generation, how many generations would it take to find the correct word? TRU-COMP 3710 Genetic Algorithms 8

Cumulative selection with randomness over multiple offspring: 1. generates a random string 2. if it doesn’t match, 2. 1 generates a new generation of mutant offspring 3. selects the single offspring most like the target 4. if it doesn’t match, go to 2. 1 COMPUTER AB CDEF GHIJKL MNOPQ RSTUVW XYZ 7 3 5 1234 5678 8 TRU-COMP 3710 FHGLJGHP FHGLJGAP FHCLJGHP FHGLTGHP. . . FHGLJGHR Genetic Algorithms [Q] What do you think of this idea? [Q] When we consider the steps 2 and 3 as one generation, how many generations would it take to find the correct word? [Q] Diversity? Cumulative improvement? Random walk? 9

Genetic Algorithm – quick view n n n Developed: USA in the 1970’s Early names: J. Holland, K. De. Jong, D. Goldberg Typically applied to: n n discrete optimization Attributed features: n n n Topics not too fast good heuristic for combinatorial problems Special Features: n n Traditionally emphasizes combining information from good parents (crossover) many variants, e. g. , reproduction models, operators TRU-COMP 3710 Genetic Algorithms 10

2. Representation n How to represent a solution (or state) for a given problem? Genetic techniques can be applied with a range of representations. Usually, we have a population of chromosomes (individuals) that are n n Usually a string of bits or numbers A complete representation of solution Each chromosome (individual) consists of a number of genes. Other representations are equally valid. Nowadays it is generally accepted that it is better to encode numerical variables directly as n n n Integers Floating point variables Permutation representation: sorting; traveling salesman problem 32752411 TRU-COMP 3710 Genetic Algorithms 11

Topics n Encoding (representation of a solution) is very important. Genotype space = {0, 1}L Phenotype space Encoding (representation) 10010001 10010010 01001 011101001 Actual observed properties TRU-COMP 3710 Decoding (inverse representation) Full hereditary information Genetic Algorithms 12

3. The Algorithm n The algorithm is used as follows: 1. 2. Generate a random population of chromosomes (, the first generation); While (the termination criteria are not satisfied) 1. 2. Each individual is a goal candidate. Determine the fitness of each chromosome (individual); Apply crossover to selected chromosomes from the current generation to generate a new population of chromosomes (the next generation); q q 3. 2. 3. 4. Fitter chromosomes have more chances to survive to the next generation. (Survival of the fittest) Each pair of parents produces two offspring and those offspring replace the parents in the next generation. Apply mutation to a new population; q 1. Topics A little bit of mutation is applied to offspring. [Q] The representation of chromosomes (individuals)? [Q] The size of the population? [Q] The crossover rate? : ~0. 8 => the parents can survive in the n. g. ? [Q] The mutation rate? : ~0. 001 TRU-COMP 3710 Genetic Algorithms 13

4. Fitness and Selection n n Fitness is an important concept in genetic algorithms. The fitness of a chromosome determines how likely it is that it will reproduce. Fitness is usually measured in terms of how well the chromosome solves some goal problem. Fitness can also be subjective. TRU-COMP 3710 Genetic Algorithms 14

n E. g. , sorting (Just for explanation. There are many good deterministic sorting algorithms. ) n [Q] Chromosome for the sorting problem of numbers? Representation of solutions? n n n If the genetic algorithm is to be used to sort numbers, then the fitness of a chromosome will be determined by how close to a correct sorting it produces. Fitness idea 1: n n n For each element in a list, count the number of right side elements that are bigger than the element; and add the counts. (0 3 2 1) -> 3 + 0 + 0 = 3; (1 0 3 2) -> ? ? ? = 4 Fitness idea 2: n n n (1 0 3 2), (0 3 2 1), . . . Count the number of pairs of adjacent elements, in which the left element is smaller than the right element. (0 3 2 1) -> 1; (1 0 2 3) -> ? ? ? [Q] Which idea looks better? TRU-COMP 3710 Genetic Algorithms 15

Selection n Roulette wheel selection 1/6 = 17% A 3/6 = 50% B Fitness(A) = 3 C Fitness(B) = 1 2/6 = 33% Fitness(C) = 2 [Q] How to implement? n n Age based selection … TRU-COMP 3710 [50 | 67(=50+17) | 100=(67+33)] Random floating point number in [0, 100) [0, 50) => A, [50, 67) => ? , [67, 100) => C Genetic Algorithms 16

var fitnesses = []; . . . var total = 0; for (var i = 0; i < fitnesses. length; i++) total += fitnesses[i]; var fitness_ratios = []; for (var i = 0; i < fitnesses. length; i++); fitness_ratios[i] = ? ? ? ; var roulette_wheel = []; roulette_wheel[0] = fitness_ratios[0]; for (var i = 1; i < fitness_ratios. length; i++) ? ? ? ; What is roulette_wheel[fi tnesses. length. TRU-COMP 3710 Genetic Algorithms 17 1]?

Topics var r; r = Math. random(); // r is in [0, 1); var i; for (i = 0; i < roulette_wheel. length; i++) ? ? ? TRU-COMP 3710 Genetic Algorithms 18

5. Crossover n Single-point crossover is applied as follows: 1. 2. 3. Select a random crossover point. Break each chromosome into two parts, splitting at the crossover point. Recombine the broken chromosomes by combining the front of one with the back of the other, and vice versa, to produce two new chromosomes. TRU-COMP 3710 Genetic Algorithm 19

n Single-point crossover n More likely to keep together genes that are near each other. n Can never keep together genes from opposite ends of string. n This is known as Positional Bias. n n n [Q] What if all the individuals have wrong genes at the left side? Can be exploited if we know about the structure of our problem, but this is not usually the case. [Q] How to solve? TRU-COMP 3710 Genetic Algorithm 20

n Idea 1: Usually, crossover is applied with one crossover point, but can be applied with more, such as in two-point crossover that has two crossover points: [Q] Can this idea solve the positional bias problem? n n Idea 2: Uniform crossover involves using a probability to select which genes to use from chromosome 1, and which from chromosome 2. Single offspring can be produced by uniform crossover … TRU-COMP 3710 Genetic Algorithm 21

Topics n n [Q] Do we always have to generate new offspring? Cloning is possible with the crossover rate of ~0. 8. [Q] what does this mean? TRU-COMP 3710 Genetic Algorithm 22

6. Mutation n n Topics A unary operator – applies to one chromosome. Randomly selects some bits (genes) to be “flipped” n 1 => 0 and 0 =>1 n Mutation is usually applied with a low probability, such as 1 in 1000. n [Q] Why is mutation necessary? ? ? n n Mutation-only-GA is possible, crossover-only-GA would not work. Crossover makes a big jump to an area somewhere “in between” two (parent) areas. Mutation creates random small diversions, thereby staying near (in the area of ) the parent. To hit the optimum you often need a ‘lucky’ mutation. TRU-COMP 3710 Genetic Algorithm 23

7. Termination Criteria n n A genetic algorithm is run over a number of generations until the termination criteria are reached. Typical termination criteria are: n n Topics Stop after a fixed number of generations. Stop when a chromosome reaches a specified fitness level. Stop when a chromosome succeeds in solving the problem, within a specified tolerance. Human judgement can also be used in some more subjective cases. TRU-COMP 3710 Genetic Algorithm 24

8. Examples n n Optimization of a mathematical function Traveling salesman problem Maze marcher Sorting TRU-COMP 3710 Genetic Algorithm 25

E. g. - optimizing a mathematical function n n A genetic algorithm can be used to find the highest integer value for f(x) = sin (x) within [0, 15]. [Q] How do you want to encode the problem? Variables? Representation of solutions? n n n [Q] How to measure fitness of a chromosome? n n n Each chromosome consists of 4 bits, to represent the values of x from 0 to 15. E. g. , (1, 0, 0, 1) 0 (f(x) = – 1) to 100 (f(x) = 1). [Q] How? Fitness(x) = 50 * (f(x) + 1) = 50 * (sin (x) + 1) By applying the genetic algorithm it takes just a few generations to find the optimal solution for f(x). TRU-COMP 3710 Genetic Algorithm 26

n The first generation n n 1001 0011 1010 0101 The fitness function n Fitness(x) = 50 * (f(x) + 1) = 50 * (sin(x) + 1), and fitness values are … Chromosome Fitness(x) Fitness ratio 1001 70. 61 0. 463 0011 57. 06 0. 374 1010 22. 80 0. 149 0101 2. 05 0. 134 n Using the random numbers between 0 and 100, and the fitness ratio, chromosomes are selected to produce offspring. [Q] HOW? ? ? n n 70. 61 / (70. 61+57. 06+22. 8+2. 05) Roulette wheel selection [Q] What if only the fittest chromosome is used to produce the next generation? TRU-COMP 3710 Genetic Algorithm 27

8. n Selection based on fitness ratios n n n 0011 1011 Crossover n n n 1001 1000 10|01 10|00 00|11 10|11 => => 1011 ? ? ? 0001 [Q] The next generation and fitness values? Chromosome Fitness(x) Fitness ratio 1011 0 0 0001 92. 07 0. 481 1011 0 0 1000 99. 47 0. 519 TRU-COMP 3710 Genetic Algorithm 28

E. g. - traveling salesman problem n Problem: • n n [Q] Is the problem a CSP? [Q] How to encode? Representation of solutions? • • n Label the cities 1, 2, … , n, and ? One complete tour is one permutation. (But, for n = 4, [1, 2, 3, 4] and [3, ? , ? ] are the same. ) Search space is really BIG: n n Given n cities, find a complete tour with the minimal length. For 30 cities there are 30! 1032 possible tours. Demo n Many web sites; http: //parano. github. io/Genetic. Algorithm-TSP/ TRU-COMP 3710 Genetic Algorithm 29

n Fitness n [Q] Do we know a minimal path? 12345 n [Q] Then, what kind of fitness function can be used? n n Let’s assume that for each pair of two cities, the shortest path is already determined. [Q] Can we determine the length of a tour? n n n TRU-COMP 3710 For (1, 2, 3, 4, 5), the length = D(1, 2) + D(2, 3) + D(3, 4) + D(4, 5) + D(5, 1) The shorter, the better. [Q] Let a, b, c, d are the lengths of 4 tours. Then fitness ratios? ? ? Genetic Algorithm 30

n Fitness n [Q] Let a, b, c, d are the lengths of 4 tours. Then fitness ratios? ? ? E. g. , a = 10, b=20, c=30, d=40 (The smaller, the better. ) n n n TRU-COMP 3710 Idea 1: Like the example of n-queens puzzle n Let x = a + b + c + d; y = (x – a) + (x – b) + (x – c) + (x – d) n The fitness ratio of a can be defined as (x – a) / y. => x = 100; y = 90 + 80 + 70 + 60 = 300 => 9/30, 8/30, 7/30, 6/30 not much distinctive Idea 2: n Let x = a^2 + b^2 + c^2 + d^2; y = (x – a^2) + (x – b^2) + (x – c^2) + (x – d^2) n The fitness ratio of a can be defined as (x – a^2) / y. => x = 3000; y = 2900 + 2600 + 2100 + 1400 = 9000 => 29/90, 26/90, 21/90, 16/90 still not much distinctive Idea 3: n Let x = 1/a + 1/b + 1/c + 1/d n The fitness ratio of a can be defined as (1/a) / x. (The larger the better) => 12/25, 6/25, 4/25, 3/25 => a has better chance than the ideas 1 and 2. Genetic Algorithm 31

n Example n n 4 cities – 1, 2, 3, 4 Distances 0 10 20 30 10 0 20 30 20 20 0 10 30 30 10 n n 4 individuals – [1, 2, 3, 4], [1, 3, 2, 4], [2, 4, 3, 1], [4, 1, 2, 3] Fitness using the Idea 3 in the previous slide? n n 0 Lengths – 70, 100, 70 x = 1/70 + 1/100 + 1/70 = 37/700 fitness 1 = (1/70)/(37/700) = 10/37, fitness 2 = 7/37, fitness 3 = 10/37, fitness 4 = 10/37 Roulette wheel? TRU-COMP 3710 Genetic Algorithm 32

n Crossover n “Normal” crossover operators will often lead to inadmissible solutions. 12345 12321 Any problem? ? ? 54321 54345 Why? ? ? n Idea 1: a) Copy randomly selected set from first parent n b) Copy rest from second parent while preserving the order in second parent [Q] The other one? ? ? TRU-COMP 3710 Genetic Algorithm 347826591 33

n Idea 2: n n Random single point crossover Other part comes from the other parent while preserving the order following the last city in the selected part 1 7 3 4 5 6 2 8 9 9 4 3 8 2 6 5 1 7 3 4 8 2 6 5 9 9 4 3 8 1 7 5 6 2 TRU-COMP 3710 Genetic Algorithm 34

n How to mutate? n Idea 1: Insert Mutation for permutations n n Idea 2: Swap mutation for permutations n n n Pick two cities at random Move the second to follow the first, shifting the rest along to accommodate Note that this preserves most of the order and the adjacency information Pick two cities at random and swap their positions Preserves most of adjacency information (4 links broken), disrupts order more Inversion Scramble. . . TRU-COMP 3710 Genetic Algorithm 35

8. n Termination criterion? TRU-COMP 3710 Genetic Algorithm 36

E. g. – maze marchers 8. TRU-COMP 3710 37 Genetic Algorithm

E. g. – sorting for experiment n Chromosome ? ? ? n n n Lists E. g. , (1 0 3 2), (0 3 2 1), . . . Fitness ? ? ? n Fitness idea 1: n n n Fitness idea 2: n n n For each element in a list, count the number of right side elements that are bigger than the element; and add the counts. (0 3 2 1) -> 3 + 0 + 0 = 3; (1 0 3 2) -> 2 + 0 + 0 = 4 Count the number of pairs of adjacent elements, in which the left element is smaller than the right element. (0 3 2 1) -> 1; (1 0 3 2) -> 1 [Q] Crossover ? ? ? [Q] Mutation ? ? ? Termination criteria ? ? ? TRU-COMP 3710 Genetic Algorithm 38

n [Q] Crossover ? ? ? n [Q] Simple crossover ? ? ? n n n [Q] Then? Idea 1: Can we use the crossover in the traveling salesman problem? n n (0 3 2 1 4) and (1 0 3 2 4) -> ? ? ? (0 3 2 4 1) and (1 0 2 4 3) // problem? Idea 2: A bit different from the above crossover, so that the elements coming from the spouse have the same order. n n (0 3 2 1 4) and (1 0 3 2 4) -> ? ? ? (0 3 3 2 4) and (1 0 2 1 4)? (0 3 2 1 4) and (1 0 3 2 4) -> ? ? ? (0 3 1 2 4) and (1 0 3 2 4) Can you evaluate them to see an improvement? Idea … [Q] Mutation ? ? ? Termination criteria ? ? ? TRU-COMP 3710 Genetic Algorithm 39

8. n Mutation ? ? ? n n Topics Idea 1: Exchange of adjacent elements that are randomly chosen (0 3 2 1 4) -> ? ? ? Mutation rate ? ? ? Termination criteria ? ? ? n Idea 1: Idea 2: Idea 3: n Idea 4: n n n TRU-COMP 3710 Until the algorithm finds the sorted list Limited number of generations Until the overall fitness reaches a certain threshold Until the difference of overall fitness becomes smaller than a threshold Combination of the above ideas Genetic Algorithm 40

9. Why does GA work well? n n n In many cases, genetic algorithms quickly produce optimal or nearoptimal solutions to combinatorial problems that would otherwise be impractical to solve. Why do genetic algorithms work well? It is possible to explain genetic algorithms by comparison with natural evolution: small changes that occur on a selective basis combined with reproduction will tend to improve the fitness of the population over time. TRU-COMP 3710 Genetic Algorithm 41

The Building Block Hypothesis n n 9. Genetic algorithms manipulate short, low-order, high fitness Topics schemata in order to find optimal solutions to problems. These short, low-order, high fitness schemata are known as building blocks. Hence genetic algorithms work well when small groups of genes that are close together represent useful features in the chromosomes. (Try to recall TSP. ) This tells us that it is important to choose a correct representation. TRU-COMP 3710 Genetic Algorithm 42

10. Non-Standard GAs n Genetic algorithms can be deceived by fit building blocks that happen not to combine to give the optimal solutions. Deception can be avoided by inversion: this involves reversing the order of a randomly selected group of bits within a chromosome. The inversion rate is ~0. 001. n Other ideas n n 1. 2. Messy GAs Co-evolution TRU-COMP 3710 Genetic Algorithm 43

Messy Genetic Algorithms n n n An alternative to standard genetic algorithms, which avoids deception. Each bit in the chromosome is represented as a (position, value) pair. For example: ((1, 0), (2, 1), (4, 0)) In this case, the third bit is undefined, which is allowed with MGAs. Underspecified bits are filled with bits taken from a template chromosome. The template chromosome is usually the best performing chromosome from the previous generation. A bit can also overspecified: ((1, 0), (2, 1), (3, 0), (4, 0)) Overspecified bits are usually dealt with by working from left to right and using the first value specified for each bit. TRU-COMP 3710 Genetic Algorithm 44

Messy Genetic Algorithms n n 10. MGAs use splice and cut instead of crossover. Splicing involves simply joining two chromosomes together: ((1, 0), (3, 0), (4, 1), (6, 1)) ((2, 1), (3, 1), (5, 0), (7, 0), (8, 0)) ((1, 0), (3, 0), (4, 1), (6, 1), (2, 1), (3, 1), (5, 0), (7, 0), (8, 0)) n Cutting involves splitting a chromosome into two: ((1, 0), (3, 0), (4, 1)) ((6, 1), (2, 1), (3, 1), (5, 0), (7, 0), (8, 0)) TRU-COMP 3710 Genetic Algorithm 45

Co-Evolution n 10. In the real world, the presence of predators is responsible for many. Topics evolutionary developments. Similarly, in many artificial life systems, introducing “predators” produces better results. Example Fish tank n n This process is known as co-evolution. For example, Ramps, which were evolved to sort numbers: “parasites” were introduced which produced sets of numbers that were harder to sort, and the ramps produced better results. TRU-COMP 3710 Genetic Algorithm 46

n Project idea n Genetic algorithm for n n n Topics The n-queens problem The class scheduling problem Local search for n TRU-COMP 3710 The traveling salesman problem Advanced Search 47