Swarm Intelligence Antbased algorithms Ref Various Internet resources
Swarm Intelligence Ant-based algorithms Ref: Various Internet resources, books, journal papers (see assignment 3 references) B. Ombuki-Berman 1
Ant-based algorithms B. Ombuki-Berman 2
Swarm Intelligence n Originated from the study of colonies, swarms of social organism n Studies of the social behavior of organisms in swarms inspired the development of very efficient algorithms q q n the foraging behaviour of ants resulted in ant colony optimization algorithms simulation studies of of bird flocks results in Particle swarm optimization A very recent and young field in computer science, with much potential q . . lots of room for contributions! B. Ombuki-Berman 3
Ant-based algorithms n Ant-based systems are a population-based stochastic search methods. q n Sound familiar- it is similar to genetic algorithms There is a population of ants, with each ant finding a solution and then communicating with the other ants, how? B. Ombuki-Berman 4
Ant System n Swarm Intelligence Algorithm q q n Based on real life animal swarms/groups Exhibit efficient ways to solve problems Ant System q q Developed by Marco Dorigo, 1991 Modeled after real life ant colonies, based on results of experiment by Goss B. Ombuki-Berman 5
Ant System n Experiment by Goss et al ’ 89 q q Ants started at nest Food placed some distance away Paths of different length between nest and food Ants found shortest path! B. Ombuki-Berman 6
Ant Algorithms for shortest path n n n Ants choose paths depending on pheromone After collecting food, paths are marked After some time, the shortest path has the highest probability B. Ombuki-Berman 7
Ant System n When ants travel they mark their path with substance called pheromone q n When an ant reaches a fork in its path the direction it follows is based on amount of pheromone it detects q n Attracts other ants Decision probabilistically made This causes positive feedback situation (i. e. Choosing a path increases the probability it will be chosen) B. Ombuki-Berman 8
Ant algorithms n We need to explore the search space, rather than simply mapping a route • Ants should be allowed to explore paths and follow the best paths with some probability in proportion to the intensity of the pheromone on a given edge/trail. n If the ants simply follow the path with the highest amount of pheromone on it, our search will quickly likely settle on a very suboptimal solution B. Ombuki-Berman 9
Ant algorithms • The probability of an ant following a certain route is a function of both the pheromone intensity, and of what the ant can see. • Furthermore, the pheromone trail must not build unbounded, hence evaporation is needed. B. Ombuki-Berman 10
Ant System n Group of ants start at home/nest q n Travel on edges q n An initial amount of pheromone already placed on edges Edges contain pheromone amount Visit nodes q Probability of Selecting next node n Based on distance between nodes and pheromone amount B. Ombuki-Berman 11
Ant System n Ants travel from node to node until end q n Once all ants finished q q n decision based on transition probability (called state transition) Solutions compared Pheromone evaporation applied to all edges Pheromone increased along each edge of best/each ant’s path n Original ant system: at each iteration, the pheromone values are updated by all the ants that have build a solution in the iteration itself. Daemon activities can be run (like local search) Redo until termination criteria met B. Ombuki-Berman 12
Requirements n Problem being solved must be in graphical format q q n Since algorithm is based on path finding behavior Not always apparent Must be finite (must have a start and end) B. Ombuki-Berman 13
Algorithm n While ( termination not satisfied ) q create ants n n q q q Starting point depends on problem constraints Initial pheromone is > 0, but very small Find solutions Pheromone update Daemon activities {optional} B. Ombuki-Berman 14
Algorithm n While ( termination not satisfied ) q create ants q Find solutions n Transition probability: Quantity of pheromone Heuristic distance α, β constants q q Pheromone update Daemon activities {optional} B. Ombuki-Berman 15
Algorithm (AS) n While ( termination not satisfied ) q create ants Find solutions Pheromone update q Daemon activities {optional} q q Pheromone laid by each ant that uses edge (i, j) Evaporation rate B. Ombuki-Berman 16
NOTE: Ant System (AS) n Pheromone update : n T is the tour done at time t by ant k, L is the length, Q is a heuristic parameter. Pheromone decay: n B. Ombuki-Berman 17
Algorithm n While ( termination not satisfied ) q q create ants Find solutions Pheromone evaporation Daemon activities {optional} n n Usually, a local search algorithm is employed here May also appear after “Find solutions” stage B. Ombuki-Berman 18
Problems n n Ant System tends to converge quickly This means that its exploitation of the best solution found is too high, it should be exploring solution space more q n Pheromone evaporation/update rule (better rule may exist) n what is the evaporation rate? Led to extensions of the ant system q q MAX-MIN Ant system Ant colony system Foot-Stepping Others (will not be discussed) B. Ombuki-Berman 19
MAX-MIN Ant System n Developed by Stutzle and Hoos 2000 n An improvement over the original Ant System to allow for more exploration q Introduced use of tending to global best from iteration best solution over time n i. e. , only the best ant updates the pheromone trails, and that, n the value of the pheromone is bound, upper and lower bound imposed q q n bounds on pheromone that are dependant on solution quality Bounds set empirically Max-min has obtained good results for many types of problems B. Ombuki-Berman 20
MAX-MIN Ant System n MMAS q q q High exploration at the beginning Only best ant can add pheromone Sometimes uses local search to improve its performance B. Ombuki-Berman 21
Ant Colony System (ACS) n Most popular/interesting contribution of ACS is n introduction of a local pheromone update in addition to the pheromone update performed at the end of the construction process (known as offline pheromone update) n Local pheromone update is performed by all ants after each construction step q Each ant applies it only to the last edge traversed: where is the pheromone decay coefficient B. Ombuki-Berman 22
Ant Colony System n n n The offline pheromone update, similary to MAX-MIN is applied to the end of each iteration, by only one ant, which can either be the iteration-best, or best-so-far. The update rule is slightly different from MAX-MIN though (see paper) Another important difference between ACS and Ant System is in the decision rule used by the ants during the construction process by using the so called pseudo random proportional rule. B. Ombuki-Berman 23
Compared to GAs n Compared to genetic algorithms: q q retains memory of entire colony instead of previous generation only less affected by poor initial solutions (due to combination of random path selection and colony memory) B. Ombuki-Berman 24
ACO for TSP n n n Input: set of cities given, and distance between each city is known Goal: Find the shortest tour that allows each city to visited exactly once. ACO algorithm set parameters, initiliaze pheromone trails while termination condition not met do Construct. Ant. Solution Apply. Local search (optional) Update. Pheromones End. While B. Ombuki-Berman 25
Ant Algorithms and the TSP • At the start of the algorithm an ant is placed in each city • Variations have been proposed by Dorigo et al. B. Ombuki-Berman 26
Ant Algorithms and the TSP • Time, t, is discrete. t(0) marks the start of the algorithm. At t+1 every ant will have moved to a new city • Assuming that the TSP is being represented as a fully connected graph, each edge has an intensity of trail on it. This represents the pheromone trail laid by the ants • Let Ti, j(t) represent the intensity of trail edge (i, j) at time t B. Ombuki-Berman 27
Ant Algorithms and the TSP • When an ant decides which city to move to next, it does so probabilistically, based on the distance to that city and the amount of pheromone intensity on the connecting edge • The distance to the next city, is known as the visibility, nij, and is defined as 1/dij, where, d, is the distance between cities i and j. B. Ombuki-Berman 28
Ant Algorithms and the TSP • At each time unit, i. e. , constructive step, evaporation takes place • The amount of evaporation, p, is a value between 0 and 1 B. Ombuki-Berman 29
Ant Algorithms and the TSP • In order to prevent ants from visiting the same city in the same tour a data structure, Tabu list, is maintained • This prevents ants from visiting cities they have already visited • Tabuk is defined as the list for the kth ant and it holds the cities that have already been visited B. Ombuki-Berman 30
Ant Algorithms and the TSP • After each ant tour the trail intensity on each edge is updated using the following formula Tij (t + n) = p. Tij(t) + ΔTij Q is a constant and Lk is the tour length of the kth ant B. Ombuki-Berman 31
Ant Algorithms and the TSP • Transition Probability where and are control parameters that control the relative importance of pheromone trail versus visibility B. Ombuki-Berman 32
Ant Systems Algorithm for TSP Initialize Place each ant in a randomly chosen city For Each Ant Choose Next. City(For Each Ant) yes more cities to visit No Return to the initial cities Update pheromone level using the tour cost for each ant No Stopping criteria yes B. Ombuki-Berman Print Best tour 33
Ant Algorithms - Applications • Marco Dorigo, who did the seminal work on ant algorithms, maintains a WWW page devoted to this subject • http: //iridia. ulb. ac. be/~mdorigo/ACO. html Check this site further information about ant algorithms, tutorial, software, applications and main publications. B. Ombuki-Berman 34
Ant Colony Optimization applied to Job Shop Scheduling Mario Ventresca and B. Ombuki-Berman 35
Overview n n n Ant System Max-Min and Foot Stepping Ant System for JSSP Brief overview of results Concluding remarks B. Ombuki-Berman 36
Job Shop Scheduling (JSSP) n n n Difficult NP-Hard Optimization Problem Involves assigning jobs to machines Jobs broken into tasks q q n Same number of tasks per job as machines Each task has processing time Task constraints and limitations q q q Processed according to predefined order No concurrent processing No-preemption B. Ombuki-Berman 37
Job Shop Scheduling Example Instance: (machine, duration) Job 1 Job 2 (2, 10) (1, 12) (1, 2) (3, 6) (3, 5) (2, 2) Possible Schedule (not optimal) M 1 M 2 M 3 Job 1 Job 2 B. Ombuki-Berman Job 2 Job 1 24 32 35 38
Recall Problems n Ant System tends to converge quickly n Extensions q q q Max-Min Ant Colony system Foot-Stepping B. Ombuki-Berman 39
Ant System and JSSP n Graphical Format Uni-directional links maintain pre-ordering criteria, distance is processing time 2 M 1 M 3 M 2 Job 1 M 3 1 M 2 3 Task B. Ombuki-Berman NOTE!! Once a task has been visited it cannot be returned to 40
Foot Stepping n An original contribution used to enhance exploration in the Ant System n In nature it would be like stepping on an already found ant path (being careful not to step on any) q Ants are forced to find a different way around, maybe (hopefully) even a better path B. Ombuki-Berman 41
Foot-Stepping n n Developed in 2003 (Ventresca, Ombuki) Allow ants to perform further search as opposed to hybridization (via local search) q n More like nature, uses ants to improve solutions by altering environment Idea q q Alter pheromone values to force ants to search more of solution space Apply when ants seem to have converged to a solution B. Ombuki-Berman 42
Foot Stepping n Two main types q q Non-Propagation based Foot Stepping (NPFS) Propagation based Foot Stepping (PFS) n NPFS works in the same manner as real life, but the foot goes away… n NPFS, Basic Algorithm: 1. 2. 3. 4. 5. Select a set of vertices cause highest pheromone leaving each vertex to become very small (not zero!!) Cause other pheromone values to increase Continue running ant algorithm until or near stagnation, reinforcing best solution Repeat desired number of times B. Ombuki-Berman 43
Foot Stepping n Propagation q q n From the original footstep location, decreasing amounts of pheromone are deposited along each proceeding vertex’s non-maximal pheromone edges Propagation is based on the solution of the best ant thus far PFS Basic Algorithm: 1. 2. 3. 4. 5. 6. Select a set V, of vertices cause highest pheromone leaving each vertex to become very small (again, not zero!!) Cause other pheromone values to increase From each vertex of V propagate pheromone to end of graph or desired distance Continue running ant algorithm until or near stagnation, reinforcing best solution Repeat desired number of times B. Ombuki-Berman 44
Summary of Results n Foot-Stepping always improved the Ant System results (unless AS found an optimal) q n Most foot-steps caused improvements Foot-Stepping was compared to Max-Min q q Both techniques seemed to yield similar results Nearly all solutions were within 2 -5% of each other B. Ombuki-Berman 45
Concluding remarks n A pheromone altering strategy (Foot Stepping) was presented that improved solution quality over a basic Ant System for the JSSP q q q n Alters the ants decision policy probabilities, thus forcing them to explore around the current solution Shows potential for leaving local optima Seems to yield similar results to Max-Min Future work involves q q Statistics to provide confidence in method Examine different propagation techniques and types problems B. Ombuki-Berman 46
- Slides: 46