Outline 1 2 General Design and Problem Solving

Outline 1. 2. General Design and Problem Solving Strategies More about Dynamic Programming – 3. Example: Edit Distance Backtracking (if there is time) – Another Strategy for the Knapsack Problem:

Design Strategies l Dynamic Programming Design Strategy – – l Solve an “easy” sub-problem Store the solution Use stored solution to solve a more difficult subproblem. Repeat until you solve the “big” hard problem Other Strategies – – – Divide and Conquer Brute Force Greedy

Design Strategies l l Dynamic Programming is not divide and conquer. Consider Floyd’s algorithm – – At no point did we break the input into two parts What is Floyd’s algorithm really doing?

Design Strategies l What is Floyd’s algorithm really doing? – – l STEP 1: Find all the shortest paths allowing a hop through vertex A, store the shortest paths STEP 2: Now, use that answer to find the shortest paths allowing a hop through vertex B The algorithm exploits what was already computed, so STEP 2 really finds the shortest paths allowing hops through vertex A and B.

More General Design Strategies l Top Down – – l See the big picture first Break it into parts Analyze each part Continue breaking down sub-parts into solvable tasks Quicksort is a classic example.

More General Design Strategies l Top Down - Quicksort – See the big picture first l – Break it into parts l – Put the pivot in the correct position and partition the list into two parts Analyze each part l – Need to put items in the correct sorted position Pick a pivot for each part… Continue breaking down sub-parts into solvable tasks l Continue recursively until sub-parts are lists of size 1

More General Design Strategies l Bottom Up – – – l Use the solution to larger and larger problems to solve the BIG problem and see the big picture Use solution to small tasks to solve larger problems Identify easily solvable tasks Mergesort is a classic example

More General Design Strategies l Bottom Up - Mergesort – Use the solution to larger and larger problems to solve the BIG problem and see the big picture l – Use solution to small tasks to solve larger problems l – Merging the final two sorted list Merging sorted lists Identify easily solvable tasks l Sorting lists of size 2

More General Design Strategies l Is Floyd’s Algorithm Top Down or Bottom Up?

More General Design Strategies l l Divide and Conquer can be both Top Down or Bottom Up Dynamic Programming tends to only be Bottom Up.

More General Design Strategies l Consider Bottom-up Strategies – Divide and Conquer usually merges two smaller sub-problems into a large problem l – (N/2 + N/2) N Dynamic Programming usually extends the solution in some way l l N-2 N-1 N_simple_version N_harder_version

More about Dynamic Programming l How does Floyd’s Algorithm extend the solution? – N-2 N-1 l – Does it consider a smaller graph and then extends the solution to a larger graph? N_simple_version N_harder_version l Does it consider a simpler shortest path problem and extend it to a more complex shortes path problem?

More about Dynamic Programming l In a graph, it is really easy to solve the shortest path problem if you do not allow any hops (intermediate vertices) – The adjacency matrix stores all the shortest paths (direct hop)

More about Dynamic Programming l l It is also easy to solve the problem if you only allow a hop through vertex x if (M[a][x] + M[x][b] < M[a][b]) then – l l update the distance O(N 2) is required to update all the cells Then, just repeat this process N times; one for each intermediate vertex. O(N 3) total time

Top Down vs. Bottom Up l Top Down – – – l Rethinking the design of existing ideas/inventions Managing projects that are underway Works really good in a Utopian world Bottom Up – – – Designing totally new ideas Putting together projects from scratch Seen more often in the real world.

Bottom-up Design l Top Down – – Lets build a flying carriage; what are the parts? Lift, propulsion, steering, etc. l Lets build a steering mechanism; what are the parts? We need a steering control – Umm? Wait, we need to know how the other parts work first. – l Lets build a lift mechanism; how do we do this? – l ? ? ? Lets build a propulsion mechanism

Bottom-up Design l Bottom UP – Discoveries: l l l – This shape produces lift A spinning propeller creates propulsion in the air Canvas with a wood frame is light enough Next Step: Perhaps we can build an stable, controllable flying thing.

Bottom-up Design l l Before we can analyze the big picture We have to – – – Look at some of the initial smaller problems See how they were solved See how they led to new discoveries

Another Dynamic Programming Algorithm l Problem: – l Find The Edit Distance Between Two Strings Solutions: – – Brute Force – O(KN) Greedy – No Optimal Algorithms yet Divide & Conquer – None discovered yet Dynamic Programming – O(N 2)

Edit Distance l l l How many edits are needed to exactly match the Target with the Pattern Target: TCGACGTCA Pattern: TGACGTGC

Edit Distance l l How many edits are needed to exactly match the Target with the Pattern Target: TCGACGT CA Pattern: T GACGTGC Three: – By Deleting C and A from the target, and by Deleting G from the Pattern

Edit Distance l Applications: – – – Approximate String Matching Spell checking Google – finding similar word variations DNA sequence comparison Pattern Recognition

Edit Distance – Dynamic Programming T C G A 0 1 2 3 4 C Optimal G T Cedit. Adistance for TG and TCG 5 6 7 8 9 1 0 1 2 3 4 5 6 G 2 1 2 3 4 5 A 2 3 2 1 2 3 C 4 3 2 Optimal edit distance for G 5 and 4 TCG 3 2 3 TGA T 6 5 4 3 4 1 G 7 6 5 4 C 8 7 6 5 T 3 7 Optimal 8 edit distance for 6 7 TG and TCGA 2 4 5 6 2 Optimal 3 4 5 edit distance for TGA and TCGA 1 2 3 4 3 2 1 2 5 4 3 2 3 3 Final Answer 4 6 5 5 3 2 3

Edit Distance int matrix[n+1][m+1]; 0 1 2 3 4 5 6 7 for (x = 0; x <= n; x++) matrix[x][0] = x; 1 0 1 2 3 4 5 6 for (y = 1; y <= m; y++) matrix [0][y] = y; 2 1 2 3 4 5 for (x = 1; x <= n; x++) 3 2 1 2 3 4 for (y = 1; y <= m; y++) 4 3 2 1 2 3 if (seq 1[x] == seq 2[y]) 5 4 3 2 1 2 matrix[x][y] = matrix[x-1][y-1]; 6 5 4 3 2 1 else matrix[x][y] = max(matrix[x][y-1] 7 6 5 4 +5 1, 4 3 2 8 7 6 5 6 matrix[x 5 5 3 1][y] + 1); return matrix[n][m]; 8 9 7 8 6 7 5 6 4 5 3 4 2 3

Edit Distance How many times is this assignment performed? How many times is this int matrix[n+1][m+1]; assignment performed? for (x = 0; x <= n; x++) matrix[x][0] = x; for (y = 0; y <= m; y++) matrix [0][y] = y; for (x = 1; x <= n; x++) for (y = 1; y <= m; y++)How many times is this comparison performed? if (seq 1[x] == seq 2[y]) matrix[x][y] = matrix[x-1][y. How many times is this 1]; assignment performed? else matrix[x][y] = max(matrix[x][y-1] + 1, matrix[x-1][y] + 1); return matrix[n][m];

Edit Distance – Dynamic Programming In the worst case, this may take n comparisons 0 T C G A 1 2 3 4 1 0 1 2 3 G 2 1 2 A 3 2 C 4 3 1 2 3 4 5 6 7 To derive the value 7, 3 2 1 2 3 4 5 6 we need to know that we 2 can 3 match 2 1 two 2 T’s 3 4 5 G 5 4 3 2 3 T 6 5 4 3 4 G 7 6 5 4 5 C 8 7 6 5 6 T n=8 To derive this value 5, we need to know that C G Twe. Ccan. Amatch two G’s after already matching two C’s and 5 6 7 previously 8 9 matching two T’s 4 5 6 7 8 2 1 2 3 4 To derive the value 6, 3 we 2 need 1 to 2 know 3 that we 4 can 3 match 2 3 two 4 C’s after matching two T’s 5 5 3 2 3

Edit Distance – Dynamic Programming T C G A C G T C A 0 1 2 5 8 1 0 G 2 1 A 3 2 1 Luckily, 2 3 we 4 can 5 match 6 7 these 8 two C’s 2 But 1 now 2 we’ve 3 4 matched 5 6 the 7 last symbol We can’t do any more matching (period!) 3 2 1 2 3 4 5 6 C 4 3 G 5 4 T 6 5 2 3 Given 2 1 our 2 previous 3 4 matches, 5 3 there 2 is 3 no 2 way 1 we 2 can 3 match 4 two A’s Thus, the edit distance is increased 4 3 2 1 2 3 G 7 6 5 4 3 2 3 4 C 8 7 6 5 5 3 2 3 T 3 4 6 7 9

Lesson to learn l l l There is no way to compute the optimal (minimum) edit distance without considering all possible matching combinations. The only way to do that is to consider all possible sub -problems. This is the reason the entire table must be considered. If you can compute the optimal (minimum) edit distance using less than O(nm) computations. Then you will be renown!