Chapter 3 Brute Force and Exhaustive Search Brute

Chapter 3 Brute Force and Exhaustive Search

Brute Force & Exhaustive Search n n Straightforward way to solve a problem, based on the definition of the problem itself; often involves checking all possibilities Pros: n n widely applicable easy good for small problem sizes Con: n often inefficient for large inputs

Brute Force Sorting n Selection sort n n Bubble sort n n n scan array to find smallest element scan array to find second smallest element etc. scan array, swapping out-of-order neighbors continue until no swaps are needed Both take Θ(n 2) time in the worst case. CSCE 411, Spring 2013: Set 2 3

Selection sort ALGORITHM Selection. Sort(A[0, . . n-1]) for i <- 0 to n-2 do min <- i for j <- i+1 to n-1 do if A[j] < A[min] min <- j swap A[i] and A[min] | 89 45 68 90 29 34 17 17 | 45 68 90 29 34 89 17 29 | 68 90 45 34 89 17 29 34 45 | 90 68 89 17 29 34 45 68 | 90 89 17 29 34 45 68 89 | 90 A 0 ≤ A 1 ≤ … ≤ Ai-1 | Ai, ……, Amin, ……, An-1 already sorted the last n-i elements C(n) є Θ(n 2) # of key swaps є Θ(n)

Bubble Sort • Compare adjacent elements and exchange them if out of order • Essentially, it bubbles up the largest element to the last position Which is bigger? ? A 0, … …, Aj <-> Aj+1, … …, An-i-1 | An-i ≤ … ≤ An-1 5

Running bubble sort

Brute Force: Bubble Sort (contd. ) ALGORITHM Bubble. Sort(A[0. . n-1]) for i <- 0 to n-2 do for j <- 0 to n-2 -i do if A[j+1] < A[j] swap A[j] and A[j+1] C(n) є Θ(n 2) 7

Brute force case studies • We saw two brute-force approach to sorting. • Let’s see brute-force to searching • How would you search for a key, K in an array A[0. . n-1]? 8

Sequential Search ALGORITHM Sequential. Search(A[0. . n-1], K) //Output: index of the first element in A, whose //value is equal to K or -1 if no such element is found i <- 0 while i < n and A[i] ≠ K do Input size: n i <- i+1 Basic op: <, ≠ if i < n return i Cworst(n) = 2 n+2 else return -1 If you knew A to be sorted in nondecreasing order, could you improve the search? 9

String matching (Control-f) Given a string of n characters (text) and a string of m (≤ n) characters (pattern), find a substring of the text that matches the pattern. Text: “nobody noticed him” Pattern: “not” N O B O D Y _ N O T I C E D _ H I M NN OO N TN T ON O TN O T T N O T t 0 … ti+j … ti+m-1 … tn-1 p 0 … pj … pm-1 p 0 should be tested with up to t? pattern text

Brute-force String Matching ALGORITHM Brute. Force. String. Matching(T[0. . n-1], P[0. . m-1]) for i <- 0 to n-m do Input size: n, m j <- 0 Basic op: = while j < m and P[j] = T[i+j] do j <- j+1 Cworst(n, m) = m(n-m+1) є O(nm) if j = m return i Cavg(n, m) є Θ(n) return -1 11

The closest-pair problem • Find the two closest planes above Manila • Find the two closest post offices in Toronto • Find the two closest records (people) in an Amazon database.

Closest-Pair by Brute-force • 13

Closest-Pair by Brute-force • expensive!

Convex set A set of points in the plane is called convex if for any two points p and q in the set, the entire line segment with the endpoints at p and q belongs to the set.

Convex envelope (hull) The convex envelope of a set S of points is the smallest convex set containing S. The convex envelope of the red set is the blue and red convex set. An extreme point of a convex set is a point that is not a middle point of any line segments.

Convex Hull Applications • In computer graphics or robot planning, a simple way to check that two (possibly complicated) objects are not colliding is to compute their convex hulls and then check if the hulls intersect • Estimate size of geographic range of a species, based on observations

Convex hull • Given a set S of n points, n > 2, on the plane. • THEOREM: The convex hull of S is a convex polygon. • The points do not lie on the same line. • The vertices are from S. • Vertices of the polygon are called extreme points. • We need to know which pairs of points need to be connected. 18

The convex hull problem Input: set S of n points Output: its convex envelope The points pi and pj are on the boundary if and only if all other points lie at the same side of the line segment between pi and pj. pi pj ax+by=c Given a point (x, y) not on this line, calculate ax+by. Is it > c? Do it for all points not on the line. Make sure that they all have the same sign. If not, then this is not a boundary line. There are n(n− 1)/2 pairs of possible lines. For each line, there are (n− 2) signs to check. The complexity is O(n 3).

Brute-Force Strengths and Weaknesses • Strengths – wide applicability – simplicity – yields reasonable algorithms for some important problems (e. g. , matrix multiplication, sorting, searching, string matching) • Weaknesses – rarely yields efficient algorithms – some brute-force algorithms are unacceptably slow – not as constructive as some other design techniques

Exhaustive search • Brute-force approach to combinatorial problems • Generated each and every combinatorial object of the problem • Select those of them that satisfy all the constraints, feasible solutions • Finding a desired object, the optimal

Traveling Salesman Problem (TSP) • Find the shortest tour through a given set of n cities that visits each city exactly once before returning to the city where it started • Can be conveniently modeled by a weighted graph; vertices are cities and edge weights are distances 22

Exhaustive Search • How can we solve TSP? Get all tours by generating all permutations of n-1 intermediate cities, compute the tour lengths, and find the shortest among them. 23

Traveling Salesman (TSP) 2 a 5 8 c b 3 7 1 d a b c d a 2+8+1+7 = 18 a b d c a 2+3+1+5 = 11 a c b d a 5+8+3+7 = 23 a c d b a 5+1+3+2 = 11 a d b c a 7+3+8+5 = 23 a d c b a 7+1+8+2 = 18 optimal

Exhaustive Search: Knapsack problem • Given n items of weights w 1, w 2, …, wn and values v 1, v 2, …, vn and a knapsack of capacity W, find the most valuable subset of the items that fit into the knapsack – A transport plane has to deliver the most valuable set of items to a remote location without exceeding its capacity How can we solve it ? 25

Knapsack problem • Brute force: Generate all possible subsets of the n items, compute total weight of each subset to identify feasible subsets, and find the subset of the largest value What is the time complexity? Ω(2 n) 26

Knapsack problem W=10 knapsack How about taking items in decreasing order of value/weight? w 1 = 7 v 1 = $42 Item 1 w 3 = 4 v 3 = $40 Item 3 w 2 = 3 v 2 = $12 Item 2 w 4 = 5 v 4 = $25 Item 4 Works for this example! subset weight value Ø 0 $0 {1} 7 $42 {2} 3 $12 {3} 4 $40 {4} 5 $25 {1, 2} 10 $54 {1, 3} 11 !feasible {1, 4} 12 !feasible {2, 3} 7 $52 {2, 4} 8 $37 {3, 4} 9 $65 {1, 2, 3} 14 !feasible {1, 2, 4} 15 !feasible {1, 3, 4} 16 !feasible {2, 3, 4} 12 !feasible {1, 2, 3, 4} 19 !feasible 27

Knapsack problem w 1 = 30 v 1 = $120 W=50 knapsack Item 1: $4/unit Item 2: $5/unit Item 3: $6/unit w 2 = 20 v 2 = $100 Item 2 w 3 = 10 v 3 = $60 Item 3 {Item 3, Item 2} = $60+$100 = $160 {Item 3, Item 1} = $60+$120 = $180 {Item 2, Item 1} = $100+$120 = $220 Doesn’t work for this example

Exhaustive Search • For both Traveling Salesman and Knapsack problems, exhaustive search gives exponential time complexity. • These are NP-hard problems, no known polynomial-time algorithm • Most famous unsolved problem in Computer Science: P vs. NP problem 29

Assignment Problem • n persons, n jobs • C[i, j] = cost of i-th person is assigned to jth job. • Find an assignment (bijection) with the minimum total cost. 30

Assignment Problem Cost matrix Job 1 Job 2 Job 3 Job 4 Person 1 9 2 7 8 Person 2 6 4 3 7 Person 3 5 8 1 8 Person 4 7 6 9 4 E. g, < 2, 3, 4, 1 > Person 1 to Job 2 Person 2 to Job 3 Person 3 to Job 4 Person 4 to Job 1. Generate all permutations of {1, 2, 3, 4}. Compute total cost. Find the smallest cost. 31

Assignment Problem C= 9 2 7 8 6 4 3 7 Complexity is Ω(n!) 5 8 1 8 7 6 9 4 < 1, 2, 3, 4 > cost = 9+4+1+4 = 18 < 1, 2, 4, 3 > cost = 9+4+8+9 = 30 And so on… 32

Summary • Brute force is a straightforward approach to solving a problem, usually directly based on the problem statement and definitions of the concepts involved. • Exhaustive search is a brute-force approach to combinatorial problems. It suggests generating each and every combinatorial object of the problem, selecting those of them that satisfy all the constraints, and then finding a desired object.