Simulated Annealing CEE 6410 David Rosenberg Learning Objectives

Simulated Annealing CEE 6410 David Rosenberg

Learning Objectives • Describe physical annealing process • Use web app to Identify effects of parameters on results • Solve a more complicated problem with Matlab • Compare to other heuristic methods CEE 6410 David Rosenberg 2

Annealing Process Amorphous (e. g. , glass) Gas Liquid st a F Slo w Crystal Decreasing energy => Cooling

Algorithm Overview (maximization) 1. Set initial alternative (x 1), system temperature (T 1), cooling schedule (T 2, T 3, …, Tn) 2. Calculate objective function value (f [x 1]) 3. Perturb xi to neighbor xi+1 and calculate f(xi+1) 4. IF f(xi+1) > f(xi), THEN accept xi+1 5. IF f(xi+1) < f(xi), THEN accept xi+1 with bolzman probability ELSE retain xi 6. Increment i, reduce temperature, go to Step 3 7. Terminate after iterations or change in f(x) CEE 6410 David Rosenberg 4

Demonstration • Maximize function with many peaks and one decision variable • https: //en. wikipedia. org/wiki/Simulated_annealing#/media/ File: Hill_Climbing_with_Simulated_Annealing. gif CEE 6410 David Rosenberg 5

Example 1. Solve using Simulated Annealing Max Z = f(x, y) = 1000 – [(x-1)2 + (y-1)2] s. t. -10 ≤ x ≤ 10 -10 ≤ y ≤ 10 • • Start at [-2, 4] Use Temperature schedule in http: //tinyurl. com/j 32 dqry CEE 6410 David Rosenberg 6

Example 2. How do the temperature schedule and number of iterations effect results? CEE 6410 David Rosenberg 7

Algorithm Improvements • Neighbor generation – only select neighbors with similar energy f(xi+1) - f(xi) ~ Ti • Restart – return to prior solution with much higher energy • Tabu – taboo list of alternatives already visited; accept neighbor when stuck at a local optimum • Adaptive SA – adapt temperature schedule during iterations CEE 6410 David Rosenberg 8

Implementations • Matlab – simulannealbnd(…) • R – several packages • And many others CEE 6410 David Rosenberg 9

Compare Heuristic Algorithms Characteristic Genetic Algorithms Explore strategies (to search new areas) • Mutation • Crossover Exploit strategies (to retain best solutions) • Elitism Number of parameters Lots Optimum CEE 6410 guaranteed NO Particle Swarm Simulated Annealing David Rosenberg 10

Conclusions • Mimic annealing of solids to solve optimization problem (Energy = objective function value) • Can embed simulation models • Main parameter is temperature cooling schedule • Lots of techniques to improve search • Lots of software implementations CEE 6410 David Rosenberg 11