Pruning Exponential growth How many leaves are there

Pruning

Exponential growth How many leaves are there in a complete binary tree of depth N? n depth = 0, count = 1 depth = 1, count = 2 depth = 3, count = 4 depth = 4, count = 8 depth = 5, count = 16 depth = N, count = 2 N n This is easy to demonstrate: n n Count “going left” as a 0 Count “going right” as a 1 Each leaf represents one of the 2 N possible N-bit binary numbers This observation turns out to be very useful in certain kinds of problems 2

Pruning n n Suppose the binary tree represents a problem that we have to explore to find a solution (or goal node) If we can prune (decide we can ignore) a part of the tree, we save effort saves 3 saves 7 saves 15 The higher up in the tree we can prune, the more effort we can save The advantage is exponential 3

Sum of subsets n Problem: n n n Example: n n n There are n positive integers, and a positive integer W Find a subset of the integers that sum to exactly W The numbers are 2, 5, 7, 8, 13 Find a subset of numbers that sum to exactly 25 We can multiply each number by 1 if it is in the sum, 0 if it is not n 2 0 0 0 0 5 0 0 0 0 1 1 1 7 0 0 1 1 1 0 0 1 1 8 13 0 0 1 13 1 0 8 1 1 21 0 1 20 1 0 15 1 1 28 0 0 5 0 1 18 1 0 13 1 1 26 0 0 12 0 1 25 4

Brute force n We have a brute-force method for solving the sum of subsets problem n n n For N numbers, count in binary from 0 to 2 N For each 1, include the corresponding number; for each 0, exclude the corresponding number Stop if we get lucky This is clearly an exponential-time algorithm It seems like, with a little cleverness, we could do better It turns out that we can use pruning to do somewhat better n But we are still left with an exponential-time algorithm 5

Binary tree representation n Suppose our numbers are 3, 8, 9, 17, 26, 39, 43, 56 and our goal is 100 We can describe this as a binary tree search As we search the binary tree, n n 3 (yes or no) 8 (yes or no) 9 (yes or no) 17 (yes or no) etc. A node is promising if we might be able to get to a solution from it A node is nonpromising if we know we can’t get to a solution When we detect a nonpromising node, we can prune (ignore) the entire subtree rooted at that node How do we detect nonpromising nodes? 6

Detecting nonpromising nodes n n n Suppose we work from left to right in the sequence 3 8 9 17 26 39 43 56 That is, we try things in the order 0 0 0 0 1 0 0 0 1 1 0 0 0. . . When we get to 0 0 0 1 0 43 we notice that even if we include all the remaining numbers (in this case, there is only one), we can’t get to 100 There is no need to try the 0 0 0 1 x numbers When we get to 1 1 1 0 0 101 we notice that we have overshot, hence no solution is possible with what we have so far We don’t need to try any of the 1 1 1 x x numbers 7

Still exponential n n Even with pruning, the sum of subsets typically requires exponential time However, in some cases, pruning can save significant amounts of time n n n Consider trying to find a subset of {23, 29, 35, 41, 43, 46, 48, 51} that sums to 100 Here, pruning can save substantial effort Sometimes, common sense can be a big help n Consider trying to find a subset of {16, 20, 28, 34, 48} that sums to 75 8

Boolean satisfaction n n Suppose you have n boolean variables, a, b, c, . . . , that occur in a logical expression such as (a or c or not f) and (not b or not d or a) and. . . The problem is to assign true/false values to each of the boolean variables in such a way as to satisfy (make true) the logical expression The brute-force algorithm is the same as before (0 is false, 1 is true, try all n binary numbers) Again, you can do significant pruning, if you think about the problem Anything you do, you will still end up with a solution that is exponential in n 9

Intractable problems n n The technical term for a problem that takes exponential time is intractable Intractable problems can only be solved for small input sizes Faster computer speeds will not help much— exponential growth is fast Bottom line: Avoid these problems if at all possible! 10

The End 11