MITM 613 Intelligent System Chapter 7 Genetic Algorithm

MITM 613 Intelligent System Chapter 7: Genetic Algorithm Abdul Rahim Ahmad

2 Chapter Seven: Genetic Algorithm 7. 1 7. 2 7. 3 7. 6 Abdul Rahim Ahmad Optimization The search space Searching the search space Genetic algorithms 7. 6. 1 The basic GA 7. 6. 2 Selection 7. 6. 3 Gray code 7. 6. 4 Variable length chromosomes 7. 6. 5 Building block hypothesis 7. 6. 6 Selecting GA parameters 7. 6. 7 Monitoring evolution 7. 6. 8 Lamarckian inheritance 7. 6. 9 Finding multiple optima 7. 6. 10 Genetic programming

3 Introduction § In computational AI, during training, an initial model is set up, its parameters are progressively refined as the training progressed. § The goal of the optimization is to minimize a cost function such as the difference, or error, between a desired output and the actual output. § Alternatively it can be maximizing another function called a fitness function. § Both approaches are equivalent, because the fitness is simply the negation of the cost. The term objective function embraces both fitness and cost. § Optimization of the objective function might mean either § minimizing the cost Abdul Rahim Ahmad § or maximizing the fitness.

4 Search Space § Solutions to search problem constitute the search space (or parameter space); one-dimensional or n-dimensional. § Each possible combination of parameter values represents a point in the search space. The extent of the search space is constrained by any limits that apply to the variables. § An example problem: The traveling salesperson problem - find the shortest route between cities, visiting each city only once. § By plotting the fitness for a two-dimensional search space, we obtain a fitness landscape (See next page). Here the two search parameters are x and y, constrained within a range of allowed values. For higher dimensions of search space a fitness landscape still exists, but is difficult to visualize. Abdul Rahim Ahmad

5 Fitness Landscape Abdul Rahim Ahmad

6 Search Space § An optimization algorithm may find peaks in the fitness landscape or valleys in the cost landscape. § Regardless of the number of dimensions, there is a risk of finding a local optimum rather than the global optimum for the function. § Global optimum - point in the search space with the highest fitness. § Local optimum - point whose fitness is higher than all its near neighbors but lower than that of the global optimum. § Fitness Landscape: § Smooth or correlated landscape – neighbouring points in the search space have similar fitness. Fitness of any point in the search space is a representative of the quality of the surrounding region. § Rugged landscape - neighbouring points have very different fitness with large numbers of local optima. Abdul Rahim Ahmad

7 Searching the Search Space § Impossible to examine all the search space, normally examine only a fraction of all possible candidates. § Choosing a starting point and then altering one or more variables in an attempt to increase the fitness or reduce the cost. § Approaches: § Based on a single candidate or a population of candidates. § Whether new candidates can be distant in the search space from the existing ones. Abdul Rahim Ahmad

8 Searching the Search Space § Based on a single candidate or a population of candidates. § Hill-climbing - maintain a single “best solution so far” which is refined until no further increase in fitness can be achieved. § Genetic algorithms - maintain a population of candidate solutions. The overall fitness of the population generally improves with each generation, although some decidedly unfit individual candidates may be added along the way. Abdul Rahim Ahmad

9 Searching the Search Space § Whether new candidates can be distant in the search space from the existing ones § hill-climbing - take small steps from the start point until they reach either a local or global optimum. To avoid missing global optimum, repeat the several times, starting from different points. § genetic algorithms and simulated annealing § begin by exploring anywhere in the search space in order to find the regions of highest fitness Abdul Rahim Ahmad § followed by exploitation - detailed search of the best regions of the search space identified during exploration. Several regions can be explored at the same time.

10 Note § Hill Climbing and Simulated Annealing is not covered § Only Genetic Algorithm is formally introduced. Abdul Rahim Ahmad

11 Genetic Algorithm § Inspired by natural evolution - process by which successive generations of animals and plants are modified so as to approach an optimum form. § Each offspring has different features from its parents, i. e. , it is not a perfect copy. If the new characteristics are favorable, the offspring is more likely to flourish and pass its characteristics to the next generation. § However, an offspring with unfavorable characteristics is likely to die without reproducing. Abdul Rahim Ahmad

12 Genetic Algorithm § Each cell of a living organism contains a set of chromosomes that define the organism’s characteristics. § Chromosomes - made up of genes, where each gene determines a particular trait such as eye color. § Genome – complete set of genetic material, a particular set of gene values. § Phenotype – describes the resulting set of traits § Each individual in the population of candidate solutions is graded according to its fitness. § The higher the fitness of a candidate solution, the greater are its chances of reproducing and passing its characteristics to the next generation. Abdul Rahim Ahmad

13 Genetic Algorithm § In order to implement a GA, the following design decisions need to be made: a) how to use sequences of numbers, known as chromosomes, to represent the candidate solutions; b) the size of the population; c) how to evaluate the fitness of each member of the population; d) how to select individuals for reproduction using fitness information (or determine which less-fit individuals will not reproduce); e) how to reproduce candidates, i. e. , how to create a new generation of candidate solutions from the existing population; Abdul Rahim Ahmad f) when to stop the evolutionary process.

14 The Basic GA § Example: Find the maximum value of a fitness function, f(x, y). We use search space variables, x and y, constrained to the 16 integer values in the range 0 – 15. § A chromosome corresponding to any point in the search space can be represented by two genes § Point (2, 6) can be represented by a chromosome § There are 16 possible values for the genes or alleles. § Position along the chromosome is the locus. Point (2, 6) has 2 loci. Abdul Rahim Ahmad § Loci are constrained to hold only binary values.

15 Basic GA § Chromosome could be represented by eight loci comprising the binary numbers 0010 and 0110, which represent the two genes: § There are 16 alleles for the genes, but only two possible values (0 and 1) for the loci. § In general, there are 2 N alleles for a binaryencoded gene N bits wide Abdul Rahim Ahmad

16 Flow chart § Basic assumptions § The initial population is randomly generated. § Individuals are evaluated according to the fitness function. § Individuals are selected for reproduction on the basis of fitness; the fitter an individual, the more likely it is to be selected. § Reproduction of chromosomes to produce the next generation is achieved by “breeding” between pairs of chromosomes using the crossover operator and then applying a mutation operator to each of the offspring. Abdul Rahim Ahmad

17 Cross over § Child chromosomes are produced by aligning two parents, picking a random position along their length, and swapping the tails with a probability Pc, known as the crossover probability. § Single point crossover for an eight-loci chromosome, where the mother and father genes are represented by mi and fi respectively, would be: § In single-point crossover the second crossover position is assumed to be the end of the chromosome. Abdul Rahim Ahmad

18 Cross over § Two point crossover where the chromosomes are treated as though they were circular, i. e. , m 1 and m 8 are neighboring loci: § chromosomes are treated as though they were circular, i. e. , m 1 and m 8 are neighboring loci Abdul Rahim Ahmad

19 Cross over § Multipoint crossover In general, multipoint crossover is also possible, provided there an even number of crossover points: § In the extreme case, each locus is considered for crossover, independently of the rest, with crossover probability Pc. This is known as uniform crossover. Abdul Rahim Ahmad

20 Mutation § Mutation involves altering the values of one or more loci. § This creates new possibilities for gene combinations that can be generated by crossover. § Mutation can be carried out in either of two ways: § The value of a randomly selected gene can be replaced by a randomly generated allele. This works for both binary and non binary chromosomes. § In a binary chromosome, randomly selected loci can be toggled, i. e. , 1 becomes 0 and 0 becomes 1. Abdul Rahim Ahmad § Individuals are selected randomly for mutation with a probability Pm. The main advantage of mutation is that it puts variety into the gene pool, enabling the GA to explore potentially beneficial regions of the search space that might otherwise be missed. This helps to counter premature convergence.

21 Validity Check § Check may be required to ensure that the chromosomes in the new generation represent valid points in the search space. § Example: § a chromosome comprising four genes, each of which can take three possible values: A, B, or C. The binary representation for each gene would require two bits, where each gene has redundant capacity of one extra value. In general, binary encoding of a gene with n alleles requires X bits, where X is log 2 n rounded up to the nearest integer. § Thus there is redundant capacity of 2 X – n values per gene. Using the binary coding A = 01, B = 10, C = 11, a binary chromosome to represent the gene combination BACA would look like this: Abdul Rahim Ahmad

22 Validity Check § A mutation that toggled the last locus would generate an invalid chromosome, since a gene value of 00 is undefined: § Defective chromosomes can also be generated by crossover. § In each case the problem can be avoided by using structured operators, i. e. , requiring crossover and mutation to operate at genes level rather than loci. Abdul Rahim Ahmad § Thus, crossover points could be forced to coincide with gene boundaries and mutation could randomly select new values for whole genes.

23 Validity Check § These restrictions would ensure the generation of valid chromosomes, but they also risk producing insufficient variety in the chromosome population. § An alternative approach is to detect and repair invalid chromosomes. § Once a defective chromosome has been detected, a variety of ways exist to repair it. § One approach is to generate “spare” chromosomes in each generation, which can then be randomly selected as replacements for any defective ones. Abdul Rahim Ahmad

24 Selection § Individuals are selected for reproduction on the basis of their fitness, i. e. , the fittest chromosomes have the highest likelihood of reproducing. § Selection determines not only which individuals will reproduce, but how many offspring they will have. The selection method can have an important impact on the effectiveness of a GA. Abdul Rahim Ahmad

25 Selection § Selection is strong if the fittest individuals have a much greater probability of reproducing than less fit ones. § Selection is said to be weak if the fittest individuals have only a slightly greater probability of reproducing than the less fit ones. § If the selection method is too strong, the genes of the fittest individuals may dominate the next generation population even though they may be suboptimal. § This is known as premature convergence, i. e. , the exploitation of a small region of the search space before a thorough exploration of the whole space has been achieved. Abdul Rahim Ahmad

26 Selection § On the other hand, if the selection method is too weak, less fit individuals are given too much opportunity to reproduce and evolution may become too slow. § This can be a particular problem during the latter stages of evolution, when the whole population may have congregated within a smooth and fairly flat region of the search space. § All individuals in such a region would have similar, relatively high, fitnesses and, thus, it may be difficult to select among them. § This can result in stalled evolution, i. e. , there is insufficient variance in fitness across the population to drive further evolution. Abdul Rahim Ahmad

27 Selection § Some alternative methods of selection, given a fitness for each member of the population, are now reviewed. § The first approach, fitness-proportionate selection, is prone to both premature convergence and stalled evolution. § The other methods are designed to counter these effects. Abdul Rahim Ahmad

28 Fitness-proportionate selection § In this method of selection, an individual’s expected number of offspring is proportional to its fitness. § The number of times an individual would expect to reproduce is, therefore, equal to its fitness divided by the mean fitness of the population. § A method of achieving this, as originally proposed by Holland, is roulette wheel selection with replacement. § The fitness of each individual is first normalized by dividing it by the sum of fitness values for all individuals in the population to yield its selection probability, Pi. Abdul Rahim Ahmad § Individuals are then imagined on a roulette wheel, with each one allocated a proportion of the circumference equal to their selection probability.

29 Fitness-proportionate selection § Individuals are selected for reproduction by spinning the notional roulette wheel. This is achieved in software by generating a random number in the range 0– 1. § From a fixed starting point, a notional pointer moves around the wheel by the fraction of a revolution determined by the random number. (e. g. 0. 29 ) was spun and individual A selected. § The pointer is reset at the origin and spun again to select the next individual. § To say that selection proceeds “with replacement” means that previously selected individuals remain available for selection with each spin of the wheel. In all, the wheel is spun N times per generation, where N is the number of individuals in the population. Abdul Rahim Ahmad

30 Roulette Wheel Selection Abdul Rahim Ahmad

31 Roulette Wheel Selection § Drawback of roulette wheel selection § High degree of variance - An unfit individual could, by chance, reproduce more times than a fitter one. Abdul Rahim Ahmad

32 Stochastic universal selection § A variant of proportional selection that overcomes this problem where unfit individual could, by chance, reproduce more times than a fitter one. § Individuals are allocated a proportion of the circumference of a wheel according to their fitness value. § Rather than a single pointer, there are N equally spaced pointers for a population size N. § All pointers are moved together around the wheel by the fraction of a revolution determined by the random number. § Thus only one spin of the wheel is required in order to select all reproducing individuals and so the method is less computationally demanding than roulette wheel selection. Abdul Rahim Ahmad

33 Stochastic universal selection Abdul Rahim Ahmad

34 Fitness-proportionate selection § Both variants of fitness-proportionate selection suffer from the risk of premature convergence and, later on, stalled evolution. Four scaling techniques are described below: § linear fitness scaling § Boltzmann fitness scaling § rank selection § truncation selection. § A fifth technique, tournament selection, is also described. These techniques are intended to counter premature convergence by slowing evolution and maintaining diversity in the early generations, and to counter stalled evolution by spreading out the selection rates for the population in the later stages. Abdul Rahim Ahmad

35 Tournament selection Abdul Rahim Ahmad § In the previous techniques for selection, an individual’s expected number of offspring is proportional to either its fitness or a scaled version of its fitness. § In Tournament Selection, individuals are selected on the basis of direct competition between them. § The most commonly used form is a binary tournament, in which two individuals are selected at random and their fitnesses evaluated. The fitter of the two is then selected for breeding. § Further pairs of individuals are then selected at random from the population of size N, which includes previously selected individuals, until N individuals have been selected for reproduction. § The fittest individuals would be expected to be selected several times, thereby increasing their probability of reproduction. § Although binary tournaments are the norm, tournaments with more than two competitors are possible.

36 The End – Thank You Abdul Rahim Ahmad