Backtracking An exhaustive search method Backtracking is useful

Backtracking § An exhaustive search method. § Backtracking is useful for problems which deal with making a sequence of decisions. § In order to apply the backtracking method, the desired solution must be expressed as a vector [x 1, x 2, … xn] where xi are chosen from some finite set Si. § Often the problem is to find a vector which satisfies a criterion function f(x 1, x 2, … xn).

Basic idea § The basic idea of the backtracking method is to build a solution vector one component at a time. § If we successfully build a component, we move on to the next component. § If we fail to build a component, we back up to work on the previous component.

The 8 -Queens Problem § Suppose you have 8 chess queens and a chess board § Can the queens be placed on the board so that no two queens are attacking each other.

8 -queens solution as a vector 1 3 5 2 …etc. Queen 3’s position Queen 2’s position Queen 1’s position

8 -queens: The brute force method for (i 1=1; i 1<=8; ++i 1) for (i 21=; i 2<=8; ++i 2). . for (i 81=; i 8<=8; ++i 8) if (!attack])i 1, i 2, . . . i 8 ])) { output ]i 1, i 2, . . . i 8 ]; stop; } write “there is no solution” Num. of possible vectors = 8 8 = 16, 777, 216 (first solutionn after 1, 299, 852 )

General recursive backtracking template for finding a solution Assume that array x[1. . n] and variable done are global. Variable done is set to false initially. void build(int k) { while (!done && more choices for x[k]) { x[k] = the next choice; if (x[1. . k] satisfies the criterion function) { if (k is the last component) done = true; else build(k+1) } } }

General Idea 1 2 3 While() { build(2); build(3); build(4); } } } 4 X build(1) While() { }

8 -queens: The backtracking method x 1 -8 1 -8 K 1 2 3 4 5 6 7 8 void build(int k) { while (!done && more choices for x[k]) { x[k] = the next choice; if (x[1. . k] satisfies the criterion function) { if (k is the last component) done = true; else build(k+1) } } } void build(int k){ int c=1; while (done==0 && c<=8) { if (!attack(k, c)) { x[k] = c; if (k==8) done = 1; else build(k+1); } c++; } }

X 1 2 DONE F T 3 4 4 k 2 c 1 2 3 4 5 build k 1 c 1 2 3 build(1); attack(1, c)-> F X[1]=1; build(2); C=2; attack(1, c)-> F X[1]=2; build(2); C=3; attack(2, c)-> T C=2; attack(2, c)-> T C=3; attack(2, c)-> F X[2]=3; build(3); C=4; attack(2, c)-> F X[2]=4; build(3); C=5; build k 2 c 1 2 3 4 5 Q Q Q 3 build main 2 1 Q attack(2, c)-> T C=2; attack(2, c)-> T C=3; attack(2, c)-> T C=4; attack(2, c)-> F X[2]=4; build(3); C=5; k 3 c 1 2 3 4 5 attack(3, c)-> T C=2; attack(3, a)-> T C=3; attack(3, c)-> T C=4; attack(3, c)-> T C=5; build k 3 c 1 2 3 4 5 attack(3, c)-> T C=2; attack(3, c)-> F X[3]=2; build(4); C=3; attack(3, c)-> T C=4; attack(3, c)-> T C=5; build k 3 c 1 2 attack(3, c)-> F X[3]=1; build(4); C=2; build k 4 c 1 2 3 4 5 attack(4, c)-> T C=2; attack(4, c)-> T C=3; attack(4, c)-> T C=4; attack(4, c)-> T C=5; build k 4 c 1 2 3 4 attack(4, c)-> T C=2; attack(4, c)-> T C=3; attack(4, c)-> F X[4]=3; DONE=T C=4;

Backtracking vs. brute force § Backtracking provides a systematic way of trying all paths until a solution is found § worst case: exhaustively tries all paths, traversing the entire state space tree. § Backtracking avoids searching entire branches by “backing up”. § When backtracking backs up, it prunes the state space tree. X X X X X

Backtracking vs. brute force Brute force XXX 1, 299, 852 State space tree 1, 299, 852 8 8 = 16, 777, 216 X X 763 Backtracking State space tree

State space tree § § Think of the search space as a tree. Root is the initial state of the problem. At each step, multiple choices lead to a branching of the tree. Solution is a sequence of choices (path) that leads from start state to a goal state

General recursive backtracking template for finding all solutions Assume that array x[1. . n] is global. void build(int k) { while (more choices for x[k]) { x[k] = the next choice; if (x[1. . k] satisfies the criterion function) { if (k is the last component) output x as a solution; else build(k+1) } } }

General recursive backtracking template for finding one best solution Assume that arrays x[n], b[n] are global. void build(int k) { while (more choices for x[k]) { x[k] = the next choice; if (x[1. . k] satisfies the criterion function) { if (k is the last component) if (x is better than b) b = x; else build(k+1) } } }

0/1 knapsack problem § Given a knapsack with maximum capacity W, and a set of n objects. § Each object i has a weight wi and a value vi. § Problem: How to pack the knapsack to achieve maximum total value of packed objects? Represent solution as a vector 3 6 7 10 Choices of component k: x[k-1]<choice <=n The criterion function: wi ≤ W 1<=i<=k Check for the last component?

Backtracking 0/1 knapsack Assume that arrays x[n+1], v[n+1], w[n+1] are global. x[0], w[0], and v[0] are initialized to 0. void build(int k) { int c = x[k-1]+1; while (c<=n) { x[k] = c; w[k] = w[k-1]+wc; v[k] = v[k-1]+vc; if (w[k]>W) {x[1~k-1] is a possible solution. } else if (c==n || w[k]==W) {x[1~k] is a possible soln. } else build(k+1); c++; } }

Knapsack problem: Example i wi vi 1 2 6 2 3 5 3 4 6 4 objects 4 5 10 W=8 ; 0 2; 6 2, 3; 11 2, 4; 12 3; 5 2, 5; 16 3, 4; 11 4; 6 3, 5; 15 5; 10

General recursive backtracking template with a bounding function Assume that arrays x[n], b[n] are global. void build(int k) { while (more choices for x[k]) { x[k] = the next choice; if (x[1. . k] satisfies the criterion function) { if (k is the last component) if (x is better than b) b = x; else if (bound(k)) build(k+1) } } }

Approximating Knapsack p=100 *. 9 p=2 p=10 p=4 knapsack Use the object with the largest v/w and the remaining capacity to compute the bound: 100*0. 9+2+10+4 = 106

Approximating Knapsack p=100 p=2 p=10 p=4 knapsack Have already included 3 objects. Have 2 objects left to choose from. What’s the most profit you can get from here?

Bounding Knapsack: Example i wi vi vi/wi 1 2 6 3 2 3 5 1. 67 3 4 4 5 6 10 1. 5 2 four objects W=8 ; 0 2; 6 18 2, 3; 11 2, 4; 12 2, 5; 16 3; 5 15 4; 6 14 5; 10 10

Coin changing problem 1. Let {a 1, a 2, . . . , an=1} be a set of distinct coin types, where each ai is an integer. 2. Each type is available in unlimited quantity. 3. The coin changing problem is to make up an exact amount C using a minimum total number of coins. Home work: Design a recursive backtracking algorithm for the coin changing problem. Note that you are required to use the general recursive backtracking template build. Moreover, you are required to use a bounding function to speed up your algorithm.