Algorithm Design Techniques Dynamic Programming Introduction Dynamic Programming

  • Slides: 12
Download presentation
Algorithm Design Techniques: Dynamic Programming

Algorithm Design Techniques: Dynamic Programming

Introduction • Dynamic Programming – Divide-and-conquer solution to a problem with overlapping subproblems –

Introduction • Dynamic Programming – Divide-and-conquer solution to a problem with overlapping subproblems – Similar to bottom-up recursion where results are saved in a table as they are computed – Typically applied to optimization problems – Typically best used when objects have a linear ordering and cannot be rearranged • http: //www. cs. sunysb. edu/~algorith/lectures-good/node 13. html • http: //www. cs. sunysb. edu/~algorith/lectures-good/node 12. html

Example: Fibonacci • Basic recursive solution F(N) = F(N-1)+F(N-2) • F 1 calculated three

Example: Fibonacci • Basic recursive solution F(N) = F(N-1)+F(N-2) • F 1 calculated three times • Instead, calculate numbers 1 ->n, store values as they are calculated • Use space instead of time F 4 F 2 F 3 F 2 F 1 F 0

Steps (CLR pg 323) 1. Characterize the structure of an optimal solution (optimal substructure)

Steps (CLR pg 323) 1. Characterize the structure of an optimal solution (optimal substructure) 2. Recursively define the value of an optimal solution 3. Compute the value of an optimal solution in a bottom-up fashion 4. Construct an optimal solution from computed information

Ordering Matrix Multiplications • Matrices – A (50 x 10), B (10 x 40),

Ordering Matrix Multiplications • Matrices – A (50 x 10), B (10 x 40), C (40 x 30), D (30 x 5) • Compute ABCD – Associative (not commutative) • Fits the ordering property – Decide how to parenthesize to achieve fewest operations

Ordering Matrix Multiplications • (A((BC)D) – BC = 10 x 40 x 30 =

Ordering Matrix Multiplications • (A((BC)D) – BC = 10 x 40 x 30 = 12, 000 operations – (BC)D = 12, 000 + 10 x 30 x 5 = 13, 500 – A((BC)D) = 13, 500 + 50 x 10 x 5 = 16, 000 • • (A(B(CD)) = 10, 500 ((AB)(CD)) = 36, 000 (((AB)C)D) = 87, 500 ((A(BC))D) = 34, 500

Ordering Matrix Multiplications N-1 • T(N) = Σ T(i)T(N-i) i=1 • Basic recursive solution

Ordering Matrix Multiplications N-1 • T(N) = Σ T(i)T(N-i) i=1 • Basic recursive solution is exponential • MLeft, Right = min. Left<=i<=Right{MLeft, i+Mi+1, Right+c. Left-1 cic. Right}

Ordering Matrix Multiplications 10, 500 A = 50 x 10 B = 10 x

Ordering Matrix Multiplications 10, 500 A = 50 x 10 B = 10 x 40 C = 40 x 30 D = 30 x 5 27, 000 20, 000 8, 000 12, 000 6, 000 0 A B C • N 2/2 values are calculated • O(N 3) running time 0 D

Optimal Binary Search Tree Word Probability • Create binary search tree to minimize a

Optimal Binary Search Tree Word Probability • Create binary search tree to minimize a . 22 N am . 18 and . 20 Σ pi(1+di) egg . 05 if . 25 the . 02 two . 08 i=1 if a two and am the egg

Optimal Binary Search Tree • Basic greedy strategy does not work • But, problem

Optimal Binary Search Tree • Basic greedy strategy does not work • But, problem has the ordering property and optimal substructure property i-1 Right • CLeft, Right = min{pi + CLeft, i-1+Ci+1, Right + Spj } j=Left j=i+1

Optimal Binary Search Tree (pg 431) a. 22 a am. 18 am and. 20

Optimal Binary Search Tree (pg 431) a. 22 a am. 18 am and. 20 and egg. 05 egg if. 25 if the. 02 the a. . am. 58 a am. . and. 56 am and. . egg. 30 and egg. . if. 35 if if. . the. 29 if the. . two. 12 two a. . and 1. 02 am am. . egg. 66 and. . if. 80 if egg. . the. 39 if if. . two. 46 if a. . egg 1. 17 am am. . if 1. 21 and. . the. 84 if egg. . two. 57 if a. . if 1. 83 and am. . the 1. 27 and. . two 1. 02 if a. . the 1. 89 and am. . two 1. 53 and a. . two 2. 15 and two. 08 two

Memoization • Maintain top-down recursion, but memoize (keep "memos" of) results allocate array for

Memoization • Maintain top-down recursion, but memoize (keep "memos" of) results allocate array for memo; initialize memo[1] and memo[2] to 1; fib(n) { if memo[n] is not zero, return memo[n]; memo[n] = fib(n-1) + fib(n-2); return memo[n]; } • http: //www. nist. gov/dads/HTML/memoize. html