Selected Topics in Evolutionary Algorithms II Pavel Petrovi

Selected Topics in Evolutionary Algorithms II Pavel Petrovič Department of Applied Informatics, Faculty of Mathematics, Physics and Informatics ppetrovic@acm. org July 10 th 2008

Evolutionary Computation Search for solutions to a problem Solutions uniformly encoded Fitness: objective quantitative measure Population: set of randomly generated solutions Principles of natural evolution: selection, recombination, mutation Run for many generations Selected Topics in Evolutionary Algorithms II, July 10 th 2008 2

Solving problems with EA Define and implement representation Define and implement objective function Design and implement initialization, mutation and recombination operators Select appropriate algorithm and selection method Setup and tune evolutionary parameters: Mutation rate Crossover rate Population size Selection parameters Termination criterion Selected Topics in Evolutionary Algorithms II, July 10 th 2008 3

EA Concepts genotype and phenotype fitness landscape diversity, genetic drift premature convergence exploration vs. exploitation selection methods: roulette wheel (fit. prop. ), tournament, truncation, rank, elitist selection pressure direct vs. indirect representations fitness space Selected Topics in Evolutionary Algorithms II, July 10 th 2008 4

Genotype and Phenotype Genotype – all genetic material of a particular individual (genes) Phenotype – the real features of that individual Selected Topics in Evolutionary Algorithms II, July 10 th 2008 5

Fitness landscape Genotype space – difficulty of the problem – shape of fitness landscape, neighborhood function Selected Topics in Evolutionary Algorithms II, July 10 th 2008 6

Population diversity Must be kept high for the evolution to advance Selected Topics in Evolutionary Algorithms II, July 10 th 2008 7

Premature convergence important building blocks are lost early in the evolutionary run Selected Topics in Evolutionary Algorithms II, July 10 th 2008 8

Premature convergence Selected Topics in Evolutionary Algorithms II, July 10 th 2008 9

Genetic drift Loosing the population distribution due to the sampling error Selected Topics in Evolutionary Algorithms II, July 10 th 2008 10

Exploration vs. Exploitation Exploration phase: localize promising areas Exploitation phase: fine-tune the solution Selected Topics in Evolutionary Algorithms II, July 10 th 2008 11

Selection methods roulette wheel (fitness proportionate selection), tournament selection truncation selection rank selection elitist strategies Selected Topics in Evolutionary Algorithms II, July 10 th 2008 12

Selection pressure Influenced by the problem Relates to evolutionary operators Selected Topics in Evolutionary Algorithms II, July 10 th 2008 13

Direct vs. Indirect Representations Selected Topics in Evolutionary Algorithms II, July 10 th 2008 14

Fitness Space (Floreano) Functional vs. behavioral Explicit vs. implicit External vs. internal Selected Topics in Evolutionary Algorithms II, July 10 th 2008 15

Evolutionary Robotics Solution: Robot’s controller Fitness: how well the robot performs Simulation or real robot Selected Topics in Evolutionary Algorithms II, July 10 th 2008 16

Fitness Influenced by Environment difficulty Task difficulty Robot’s abilities (sensors, actuators) Controller abilities Robot Morphology T Incremental change during evolution: Incremental Evolution Selected Topics in Evolutionary Algorithms II, July 10 th 2008 17

Evolvable Tasks Wall following Obstacle avoidance Docking and recharging Artificial ant following Box pushing Lawn mowing Legged walking T-maze navigation Foraging strategies Trash collection Vision discrimination and classification tasks Target tracking and navigation Pursuit-evasion behaviors Soccer playing Navigation tasks Selected Topics in Evolutionary Algorithms II, July 10 th 2008 18

Evolutionary algorithms Genetic algorithm Genetic programming Evolutionary Strategies Evolutionary Programming Classifier systems Ant-colony optimisation Memetic algorithms Artificial Immune Systems Selected Topics in Evolutionary Algorithms II, July 10 th 2008 19

Example: Travelling Salesman Problem (TSP) Finding a closed path that visits all cities Difficult problem (NP-complete) Selected Topics in Evolutionary Algorithms II, July 10 th 2008 20

Example: Travelling Salesman Problem (TSP) Trivial representation: ( 4, 1, 7, 2, 5, 3, 6 ) - list of cities visited Representation is a permutation, however standard crossover results in descendants that are not permutations Not suitable for standard recombination Need a different representation or recombination! Selected Topics in Evolutionary Algorithms II, July 10 th 2008 21

TSP Example: Partially matched crossover (PMX) 2 sites picked, intervening section specifies “cities” to interchange between parents: A = 9 8 4 | 5 6 7 | 1 3 2 10 B = 8 7 1 | 2 3 10 | 9 5 4 6 A’ = 9 8 4 | 2 3 10 | 1 6 5 7 B’ = 8 10 1| 5 6 7 | 9 2 4 3 some ordering information from each parent is preserved, and no infeasible solutions are generat Selected Topics in Evolutionary Algorithms II, July 10 th 2008 22

TSP Example: Order Crossover (OX) 2 sites picked, intervening section specifies “cities” to interchange between parents: A = 9 8 4 | 5 6 7 | 1 3 2 10 B = 8 7 1 | 2 3 10 | 9 5 4 6 B* = 8 H 1 | 2 3 10 | 9 H 4 H B** = 2 3 10 | H H H | 9 4 8 1 B’ = 2 3 10 | 5 6 7 | 9 4 8 1 A’ = 5 6 7 | 2 3 10 | 1 9 8 4 Order crossover preserves more information about RELATIVE ORDER than does PMX, but less about ABSOLUTE POSITION of each “city” (for TSP example) Selected Topics in Evolutionary Algorithms II, July 10 th 2008 23

TSP Example: Operator MPX 2 sites picked, intervening section specifies “cities” to interchange between parents: A = 9 8 4 | 5 6 7 | 1 3 2 10 B = 8 7 1 | 2 3 10 | 9 5 4 6 C = 5 7 1 | 2 3 10 | 9 8 6 4 D = 6 4 1 | 2 3 10 | 9 5 7 8 C' = 5 | 5 6 7 | 7 1 | 2 3 10 | 9 8 6 4 D' = 6 4 1 | 2 3 10 | 9 5 | 5 6 7 | 7 8 C'' = * | 5 6 7 | * 1 | 2 3 10 | 9 8 * 4 C''' = 5 6 7 1 2 3 10 9 8 4 Selected Topics in Evolutionary Algorithms II, July 10 th 2008 24

TSP Example: Cyclic Crossover CX Cycle crossover forces the city in each position to come from that same position on one of the two parents: A = 9 8 2 1 7 4 5 10 6 3 B = 1 2 3 4 5 6 7 8 9 10 A' = 9 - - - - 9 --1 -----9 --1 -4 --69 2 - 1 - 4 - 8 6 10 A'' = 9 2 3 1 - 4 - 8 6 10 = 9 2 3 1 7 4 5 8 6 10 A''' = 9 2 3 1 5 4 7 8 6 10 Selected Topics in Evolutionary Algorithms II, July 10 th 2008 25

Multiple-objective optimisation Several objectives to optimize Usually no single optimal solution Decision maker selects a solution from finite set by making compromises First MOEAs in mid 80 s, since then huge number of papers on EMOO EAs are good for MOO: • Inherently parallel • Less susceptible to the shape or continuity of MO search space Selected Topics in Evolutionary Algorithms II, July 10 th 2008 26

Multiple-objective optimisation Selected Topics in Evolutionary Algorithms II, July 10 th 2008 27

Multiple-objective optimisation Selected Topics in Evolutionary Algorithms II, July 10 th 2008 28

Multiple-objective optimisation Pcurrent(t) Pknown(t) Ptrue(t) Selected Topics in Evolutionary Algorithms II, July 10 th 2008 29

Multiple-objective optimisation MOEA is an extension on an EA in which two main issues are considered: • How to select individuals such that nondominated solutions are preferred over those which are dominated • How to maintain diversity as to be able to maintain in the population as many elements of the Pareto optimal set as possible. Selected Topics in Evolutionary Algorithms II, July 10 th 2008 30

Multiple-objective optimisation Preference of nondominated solutions: • All non-dominated individuals get the same probability to reproduce • This probability is higher than the one corresponding to the individuals which are dominated = PARETO RANKING Selected Topics in Evolutionary Algorithms II, July 10 th 2008 31

Multiple-objective optimisation Maintaining diversity: • Fitness sharing • Niching • Clustering • Geographically-based schemes to distribute solutions • Use of entropy Selected Topics in Evolutionary Algorithms II, July 10 th 2008 32

Multiple-objective EAs Aggregating functions • combining objectives into single fitness: • cannot generate non-convex portions of the Pareto front regardless of the weight combination used Selected Topics in Evolutionary Algorithms II, July 10 th 2008 33

Multiple-objective EAs Population-based approaches • concept of Pareto dominance is not directly incorporated into the selection process • population of an EA is used to diversify the search VEGA = Vector Evaluated Genetic Algorithm • At each generation, a number of sub-populations are generated by performing proportional selection according to each objective function in turn • Problem: selection scheme is opposed to the concept of Pareto dominance Selected Topics in Evolutionary Algorithms II, July 10 th 2008 34

Multiple-objective EAs Pareto-Based Approaches • Goldberg's Pareto Ranking • Multi-Objective Genetic Algorithm (MOGA) • The Nondominated Sorting Genetic Algorithm (NSGA) • NSGA II = NSGA + elitism & crowded comparison operator (makes the search faster) • Niched Pareto Genetic Algorithm (NPGA) – tournament • Strength Pareto Evolutionary Algorithm (SPEA) – special clustering method to maintain diversity • SPEA 2 – different clustering method (nearest neighbor) • many other. . . Selected Topics in Evolutionary Algorithms II, July 10 th 2008 35

Neuroevolution through augmenting topologies (NEAT) The most successful method for evolution of artificial neural networks Sharing fitness Starting with simple solutions Global counter i. e. Topological crossover – very important for preserving evolved structures Selected Topics in Evolutionary Algorithms II, July 10 th 2008 36

GECCO Contest GECCO is the largest EA conference (European alternative: PPSN) Humies awards Contest tasks with prizes. . . Selected Topics in Evolutionary Algorithms II, July 10 th 2008 37

Further information. . . Conferences: GECCO, PPSN, CEC (now part of WCCI, Evo. Workshops, EA) Journals: Evolutionary Computation, Genetic Programming and Evolvable Machines, IEEE Transactions on Evolutionary Computation Scientific body: ACM SIGEVO, with newsletter Mailing list: ec-digest with archive: http: //ec-digest. research. ucf. edu/ Recent publication about GP: Riccardo Poli, William B Langdon, Nicholas Freitag Mc. Phee: A Field Guide to Genetic Programming http: //www. lulu. com/content/2167025 Selected Topics in Evolutionary Algorithms II, July 10 th 2008 38