Local Search walksat ant colonies and genetic algorithms

Local Search: walksat, ant colonies, and genetic algorithms

Tonight n n n Course evaluations Local search Final exam

Issue n How to search really large spaces n n n 10^30, 000 + states systematic search hopeless! Local search n n n iterative repair, hill climbing, gradient descent, … guess a start state repeat until satisfied: make small change n move to a local neighbor

Examples n Learning weights in a neural network n n n Learning CPT’s in a Bayesian network n n Space: set of weights Neighborhood: small delta EM algorithm These are optimization problems… what about local search for decision problems?

N-Queens n N-Queens Demo

GSAT Guess a random truth assignment while (#unsat clauses > 0){ flip a variable that minimizes number of unsatisfied clauses }

Room for improvement n Less dramatic performance on structured problems (e. g. planning) n n Too greedy – can become stuck in local minima Solution: allow some “uphill” moves – many different strategies possible… n n n Simulated annealing Tabu lists Alternate GSAT + random flips

Random Walk Guess a random truth assignment while (#unsat clauses > 0){ pick an unsat clause flip any variable in the clause } n n Suppose clauses are of length 2; what is probably a flip is “correct”? Solves 2 -SAT in O(n^2) time!

Greediness + Randomness n n If clause length > 2, then pure random walk is very unlikely to converge Suppose we random walk greedily?

Walksat Guess a random truth assignment while (#unsat clauses > 0){ pick an unsat clause with probability p { flip a variable that minimizes number of newly-unsatisfied clauses } otherwise { flip any variable in clause }

Walksat results n Best known method for many problems n n graph coloring (certain) planning problems hard random problems For many (all? ) other domains, can mix random walk & backtrack search n n run backtrack DPLL until time out restart with new random seed for choosing branching variables

Stochastic Backtrack Search n Quasigroup Completion Demo

Parallel local search n n Techniques considered so far only look at one point on the search space at a time… Easy to run many copies in parallel with different random seeds n n Called portfolio approach Often gives super-linear speedup!

Run time distributions n n Often there is a great variability in run time depending on random seeds… As parallel processes are added, probability of getting a “lucky” short run can increase quickly!

Informed parallel search n Can we further improve performance by exchanging information between the parallel processes? n n Genetic algorithms Ant colony algorithms

Genetic algorithms n Idea: the genetic code of an individual is a point in the search space n n a gene = a dimension in the search space Reproduction = local moves n n Asexual reproduction (mutation) = create a child by a small change in the parent Sexual reproduction = create a child by mixing genes of two parents

The GA Cycle of Reproduction reproduction children modified children parents population modification evaluated children deleted members discard 19 evaluation

A Simple Example The Traveling Salesman Problem: Find a tour of a given set of cities so that ¨ each city is visited only once ¨ the total distance traveled is minimized 20

Representation is an ordered list of city numbers known as an order-based GA. 1) London 2) Venice 3) Dunedin 4) Singapore 5) Beijing 7) Tokyo 6) Phoenix 8) Victoria City. List 1 (3 5 7 2 1 6 4 8) City. List 2 (2 5 7 6 8 1 3 4) 21

Crossover combines inversion and recombination: * * Parent 1 (3 5 7 2 1 6 4 8) Parent 2 (2 5 7 6 8 1 3 4) Child (5 8 7 2 1 6 3 4) This operator is called the Order 1 crossover. 22

Mutation involves reordering of the list: Before: * * (5 8 7 2 1 6 3 4) After: (5 8 6 2 1 7 3 4) 23

TSP GA Demo l http: //cs. felk. cvut. cz/~xobitko/ga/tspexa mple. html 24

TSP Example: 30 Cities 25

Solution i (Distance = 941) 26

Solution j(Distance = 800) 27

Solution k(Distance = 652) 28

Best Solution (Distance = 420) 29

Overview of Performance 30

Practical Applications n n Design of small sorting circuits Optimization of code fragments n n Training neural networks n n n Used in Windows kernel? ? ? Can outperform backprop Airport scheduling (Generally) unanswered questions: n n Would other local search techniques work just as well? Is crossover really key?

Ant colony optimization n Idea: n n n Each ant in a colony is local search process Ants (processes) communicate by laying down phenoromes at visited states (“this place looked good to me”) When deciding where to move, ants prefer (with some probability) to follow trail left by other ants n Phenorome eventually “evaporates”

Real ants

Why might it work? n More “intelligent” noise gradient Ant B @ t 10 not gradient, but a good alternative Ant A @ t 3 gradient

Beyond phenomerones n Combine local gradients to get more global view gradient Ant B @ t 10 move by weighted sum of B’s own gradient and A’s gradient A @ t 10 gradient

Does it work? n n n Many successful applications in engineering & science Appears to outperform single-thread local search procedures for quadratic programming More work needed to determine exactly why metaphor works when it does!

Summary n Local search is an important tool for large, complex optimization and decision problems n n n Ideas: Iterative repair Noise to escape local optima Much work on parallel local search n n n Portfolios Genetic algorithms Ant colony optimization

So…. n n n End of the course. Is that all there is? ? ? Where was the artificial intelligence? ? !!

In my view… n AI is the study of general problemsolving strategies “Meta” algorithms 101 n Life: one damned problem after another n Strategies evolve at many levels n n Genetic – “real” evolution Individual human – logical thinking Society – spread of innovation

Unity n A relatively small number of concepts n n n various state-space search strategies syntactic structure (both formal logic & natural language) learning strategies – minimizing entropy, error, complexity arise over and over again in biological and artificial systems

Why? n This unity of thought n biology n computation is a cause for wonder … n n Work in AI is one way of working on the ultimate questions n Why? Why here? Why now? Why us?

As the Dali Lama once said… n “But if anything I have said is not helpful to you… … then just forget about it!”