Sisteme de programe pentru timp real Universitatea Politehnica

Sisteme de programe pentru timp real Universitatea “Politehnica” din Bucuresti 2005 -2006 Adina Magda Florea http: //turing. cs. pub. ro/sptr_06

Curs Nr. 11 1. Hybrid GA-heuristic search strategy 2. Learning decision rules by GA 3. Genetic programming 2

1. Hybrid GA-heuristic search strategy • A simple resource planning application • Solve it using GA first then solve it using hybrid GAheuristic search • 4 resources: A, B, C and D • Each resource carry out a number of different operation types O 1, O 2 and O 3. • For each resource: - the time to carry out an operation type - the cost of running an operation - the available capacity 3

1. 1 Using GA Given: 3 requirements (jobs) for: 1400 units of O 1, 1200 units of O 2 and 900 units of O 3 Find: the optimal resource planning that meets the following objectives: • Cheapest production cost • Delivering the required units of each job • Not exceeding the available capacity for each resource Resource O 1 O 2 O 3 Cost/min Capacity A 2 5 1 30 1000 B - 4 2 40 4720 C 1 - - 50 400 D 1 - 2 20 1000 4

Individuals = the number of units of each operation assigned to each resource. No. O 1 A No. O 2 A No. O 3 A No. O 2 B No. O 3 B No. O 1 C No. O 1 D No. O 3 D Fitness function: cost function which consists of the total cost of running the resources plus the total penalty points for exceeding the available capacity of resources. Fitness = No. O 1 A*2*30 + No. O 2 A*5*30 +. . + *(No. O 1 A + No. O 2 A + No. O 3 A – 1000 +…) – to minimize • - a value which reflects the severity of the penalty constraint. • This representation: aprox 60 generations of 50 individuals to evolve a solution which has a cost of 99% of the known optimal cost. • This is quite slow considering the simplicity of the problem. Resource O 1 O 2 O 3 Cost/min Capacity A 2 5 1 30 1000 B - 4 2 40 4720 C 1 - - 50 400 D 1 - 2 20 1000 5

1. 2 Using hybrid GA-heuristic Individuals: the sequence of jobs to be presented to a simple resource planner. First break down the three large jobs (requirements) into smaller jobs. The jobs are broken down as follows: • The job of 1400 O 1 was broken down into 5 jobs of 280 O 1 each • The job of 1200 O 2 was broken down into 4 jobs of 300 O 2 each • The job of 900 O 3 was broken down into 3 jobs of 300 O 2 each § Use 12 sequence genes to represent the order in which these jobs are presented to a simple planner. § The simple planner uses very simple heuristics: take the next job in the queue and allocate to the first resource (from A to D) which has available capacity. § This new representation strategy needs only 2 generations of 50 individuals to evolve a solution with a cost of 100% of the known optimal cost. § The evolution time was reduced from minutes in the initial strategy to seconds using the new methodology. 6

1. 3 Application • Methodology used to build an application for Channel 4 - the largest commercial TV station in the UK. • Channel 4 had the requirement of optimising the sequencing of advertisements within a commercial break. • They have to ensure that different copies (versions) of the same advert go out simultaneously to different regions of the UK whilst satisfying as many of advertisers requests for special positions as possible. • Advertisers can specify a mix of regions, first or last position in a break or a 'top & tail' pair in the same break. • A break sequencer has to ensure that all the requirements are met without overlaps or gaps in the break transmitted to each region. • In a four minute break across 6 regions there are 144 ten second slots to schedule with up to 50 adverts. • A heuristic knowledge based system solution to the problem was rather difficult to achieve and a GA optimisation approach was considered. 7

Application • GA approach: use 144 index genes to represent each of the 10 second slots across the six regions. • Each index gene returns a pointer to an advert (one of 50). • This strategy could take hours to optimise. • Given that there are over 70 breaks a day to optimise, the evolution time is unacceptably high. • GA-heuristic: use sequence of genes to represent the sequence of adverts to be scheduled (up to 50) by a simple heuristic scheduler. • This approach reduces the evolution time to seconds. • The simple scheduler works by taking the next advert in the sequence and placing it in the first available position (across the required regions) in the break. • The optimisation task becomes one of optimising the sequence in which the adverts are presented to the simple scheduler. • Major advantage: a simple scheduler would always produce a valid schedule which satisfies the hard constraints while the soft constraints are represented in the cost function and are satisfied through natural selection. 8

2. Learning decision rules by GA • Rules of the form: if A 1=v 1 and x 1<A 2<=v 2 … then Class C 1 • Both discrete and continuous values attributes • Learning set E={e 1, . . e. M} • e E is described by N atributes A 1, . . AN and labeled by a class c(e) C • Discrete value attributes – finite set V(Ai) • Continuous value attributes – an interval V(Ai) = [li, ui] 9

2. 1 Decision rules • • • A class ck C Positive examples - E+(ck) = {e E | C(e)=ck} Negative examples - E-(ck) = E – E+(ck) A decision rule R if t 1 and t 2. . and tr then ck LHS RHS – class membership of an example Rule set - RS : disjunctive set of decision rules with the same RHS CRS C – denotes the class on the RHS of an RS The GA is called for each class ck C to find the RS separating E+(ck) from E-(ck) Fitness function -> prefers rules consisting of fewer conditions, which cover many ex+ and very few ex 10

2. 2 Representation • One chromosome encodes an RS • The no. of rules in RS is not knwon Use variable length chromosomes + provide operators which change the no. of rules • 1 chromosome = concatenation of strings • Each fixed-length string = the LHS of one decision rule (no need for RHS) • 1 string is composed of N sub-strings (LHS) – a condition for each attribute 11

Representation • • Discrete value attributes – binary flags Continous value attributes – li, ui, li<Ai<=ui (+ , - ) li, ui are selected from a finite set of boundary thresholds Boundary threshold = midpoint between a successive pair of examples in the sequence sorted by increasing value of Ai, that one is ex+ and the other ex- • ex+ ex+ ex- ex- ex+ ex- exthik-1 thik+1 • If a condition is not present – li= - , ui= + Ai 12

Representation Example • 2 cont val attr: Salary, Amount • 1 disc val attr – Purpose (car, house, school) • Class - Accept Salary - + 100 250 Amount - 250 - 500 750 + - + Purpose 1 1 1 if Amount<250 then ACCEPT 1 1 1 if 100<salary<250 and Amount <500 then ACCEPT 1 1 0 if Salary>750 and Purpose = (car or house) then ACCEPT 13

2. 3 Genetic operators 4 operators applied to a single rule set: • Changing condition (a) • Positive example insertion (b) • Negative example removal (c) • Rule drop (d) 2 operators applied with 2 arguments to RS 1 and RS 2: • Rule copy (e) • Crossover (f) 14

Genetic operators (a) Changing condition • A mutation like operator – alters a single condition related to an attribute Ai • If Ai disc – randomly chooses a flag and flip • If Ai cont – randomly replaces a threshold (li or ui) by a boundary threshold (b) Pos ex insertion • Modifies a single dec rule R in RS to allow to cover a new random e+ E+(CRS) currently uncovered by R • All conditions in the rule, which conflict with e+ have to be altered. • Ai disc – flag set • Ai cont – li<Ai<=ui with ui<Ai(ex+) – smallest ui’ such as ui’>=Ai; similar if li>= Ai(ex+) 15

Genetic operators (c) Negative ex removal • The negative example removal operator alters a single rule R from the ruleset RS. • It selects at random a negative example e- from the set of all the negative examples covered by R. • Then it alters a random condition in R in such a way, that the modied rule does not cover e-. • If the chosen condition concerns a discrete attribute Ai the flag which corresponds to Ai(e-) is cleared. • If Ai is a continuous-valued attribute then the condition li < Ai <= ui is narrowed down either to li’ < Ai <= ui or to li < Ai <= ui’, where li’ is the smallest boundary threshold such that Ai(e-) >= li’ and ui’ is the largest boundary threshold such that ui’ < Ai(e-). 16

Genetic operators Rule drop and rule copy operators are the only ones capable of changing the number of rules in a ruleset. (d) Rule drop • The single argument rule drop removes a random rule from a ruleset RS. (e) Rule copy • Rule copy adds to one of its arguments RS 1, a copy of a rule selected at random from RS 2, provided that the number of rules in RS 1 is lower than max. R. • max. R is an user-supplied parameter, which limits the maximal number of rules in the ruleset. (f) Crossover • The crossover operator selects at random two rules R 1 and R 2 from the respective arguments RS 1 and RS 2. Then it applies a crossover to the strings representing R 1 and R 2. 17

2. 4 Fitness • Uses rank-based fitness asignment • • • Goal = reduction of the no. of errors ERS – the set of ex covered by the RS (class of RHS – CRS) E+RS = ERS E+(CRS) – set of ex+ correctly classified by RS E-RS = ERS E-(CRS) – set of ex- covered by RS The total no. of ex+ and ex. POS = |E+(CRS)| NEG=|E-(CRS)| = M – POS • The RS correctly classifies: - pos = |E+RS| ex+ and - NEG-neg ex- where neg=|E-RS| 18

Fitness • • • Ferror = Pr(RS) / Compl(RS) Pr(RS) = the probability of classifying correctly an example from the learning set by RS Compl(RS) = the complexity of RS Pr(RS) = (pos + NEG – neg) / (POS+NEG) Compl(RS) = (L/N+1) L – total no. of conditions in RS N – no. of attributes - user supplied in [0. 001. . 0. 1] We are interested in maximizing the probability and minimizing the complexity – to obtain a compact rule set and acorrect classification 19

2. 5 Optimisation strategy to learning decision rules ID 3 Learn order of attributs Perform ID 3 on that order 20

3. Genetic programming is a branch of genetic algorithms Functional programming Genetic programming, uses four steps to solve problems: 1) Generate an initial population of random compositions of the functions and terminals of the problem (computer programs). 2) Execute each program in the population and assign it a fitness value according to how well it solves the problem. 3) Create a new population of computer programs. i) Copy the best existing programs ii) Create new computer programs by mutation. iii) Create new computer programs by crossover 4) The best computer program that appeared in any generation, the best-so-far solution, is designated as the result of genetic programming 21

Genetic programming 22

Fitness function • The most difficult and most important concept of genetic programming is the fitness function. • It varies greatly from one type of program to the next. • Ex: create a genetic program to set the time of a clock • The fitness function - the amount of time that the clock is wrong. • Few problems have such an easy fitness function; most cases require a slight modification of the problem in order to find the fitness. • • • Gun Firing Program Training a genetic program to fire a gun to hit a moving target. Fitness function - the distance that the bullet is off from the target. The program has to learn to take into account a number of variables, such as wind velocity, type of gun used, distance to the target, height of the target, velocity and acceleration of the target. This problem represents the type of problem for which genetic programs are best. It is a simple fitness function with a large number of variables. 23

Fitness function • • • Water Sprinkler System Consider a program to control the flow of water through a system of water sprinklers. The fitness function is the correct amount of water evenly distributed over the surface. Unfortunately, there is no one variable encompassing this measurement. Thus, the problem must be modified to find a numerical fitness. One possible solution is placing water-collecting measuring devices at certain intervals on the surface. The fitness could then be the standard deviation in water level from all the measuring devices. Another possible fitness measure could be the difference between the lowest measured water level and the ideal amount of water. Maze Solving Program If one were to create a program to find the solution to a maze, first, the program would have to be trained with several known mazes. The ideal solution from the start to finish of the maze would be described by a path of dots. The fitness in this case would be the number of dots the program is able to find. In order to prevent the program from wandering around the maze too long, a time limit is implemented along with the fitness function. 24

Functions and terminals • The terminal and function sets are also important components of genetic programming. • The terminal set consists of the variables and constants of the programs. • In the maze example, the terminal set would contain three commands: forward, right and left. • The function set consists of the functions of the program. • In the maze example the function set would contain: – If "dot" then do x else do y. • In the gun firing program the terminal set would be composed of the different variables of the problem. • Some of these variables could be the velocities and accelerations of the gun, the bullet and target. • The functions are several mathematical functions, such as addition, subtraction, division, multiplication and other more complex functions. 25

Selection 1) Use probability based on the fitness of the solution. If f(Si(t)) is the fitness of the solution Si and is the total sum of all the members of the population, then the probability that the solution Si will be copied to the next generation is 2) Use tournament selection 26

Crossover • The creation of the offsprings from the crossover operation is accomplish by deleting the crossover fragment of the first parent and then inserting the crossover fragment of the second parent. • The second offspring is produced in a symmetric manner. 27

Crossover An important improvement that genetic programming displays over genetic algorithms is its ability to create two new solutions from the same solution. 28

Mutation Two types of mutations are possible. • In the first kind a function can only replace a function or a terminal can only replace a terminal. • In the second kind an entire subtree can replace another subtree. 29

Benefits • Genetic programming works best for several types of problems. • The first type is where there is no ideal solution: for example, a program that drives a car. • There is no one solution to driving a car. Some solutions drive safely at the expense of time, while others drive fast at a high safety risk. • Therefore, driving a car consists of making compromises of speed versus safety, as well as many other variables. In this case genetic programming will find a solution that attempts to compromise. • Second, genetic programming is useful in finding solutions where the variables are constantly changing. In the previous car example, the program will find one solution for a smooth concrete highway, while it will find a totally different solution for a rough unpaved road. 30