Introduction to Algorithms 6 046 J Lecture 2

Introduction to Algorithms 6. 046 J Lecture 2 Prof. Shafi Goldwasser

Solving recurrences • The analysis of integer multiplication from Lecture 1 required us to solve a recurrence. • Recurrences are a major tool for analysis of algorithms • Today: Learn a few methods. • Lecture 3: Divide and Conquer algorithms which are analyzable by recurrences. L 2. 2

Recall: Integer Multiplication • Let X = A B and Y = C D where A, B, C and D are n/2 bit integers • Simple Method: XY = (2 n/2 A+B)(2 n/2 C+D) • Running Time Recurrence T(n) < 4 T(n/2) + 100 n How do we solve it? L 2. 3

Substitution method The most general method: 1. Guess the form of the solution. 2. Verify by induction. 3. Solve for constants. Example: T(n) = 4 T(n/2) + 100 n • [Assume that T(1) = (1). ] • Guess O(n 3). (Prove O and separately. ) • Assume that T(k) ck 3 for k < n. • Prove T(n) cn 3 by induction. L 2. 4

Example of substitution T ( n) 4 T ( n / 2) 100 n 4 c( n / 2)3 100 n (c / 2) n 3 100 n cn 3 ((c / 2) n 3 100 n ) desired – residual 3 desired cn whenever (c/2)n 3 – 100 n 0, for example, if c 200 and n 1. residual L 2. 5

Example (continued) • We must also handle the initial conditions, that is, ground the induction with base cases. • Base: T(n) = (1) for all n < n 0, where n 0 is a suitable constant. • For 1 n < n 0, we have “ (1)” cn 3, if we pick c big enough. This bound is not tight! L 2. 6

A tighter upper bound? We shall prove that T(n) = O(n 2). Assume that T(k) ck 2 for k < n: T ( n) 4 T ( n / 2) 100 n cn 2 for no choice of c > 0. Lose! L 2. 7

A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) c 1 k 2 – c 2 k for k < n. T ( n) 4 T ( n / 2) 100 n 4(c 1 (n / 2) 2 c 2 (n / 2)) 100 n c 1 n 2 2 c 2 n 100 n c 1 n 2 c 2 n (c 2 n 100 n) c 1 n 2 c 2 n if c > 100. 2 Pick c 1 big enough to handle the initial conditions. L 2. 8

Recursion-tree method • A recursion tree models the costs (time) of a recursive execution of an algorithm. • The recursion tree method is good for generating guesses for the substitution method. • The recursion-tree method can be unreliable, just like any method that uses ellipses (…). • The recursion-tree method promotes intuition, however. L 2. 9

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: L 2. 10

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: T(n) L 2. 11

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: n 2 T(n/4) T(n/2) L 2. 12

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: n 2 (n/4)2 T(n/16) T(n/8) (n/2)2 T(n/8) T(n/4) L 2. 13

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: n 2 (n/4)2 (n/8)2 (n/4)2 … (n/16)2 (n/2)2 (1) L 2. 14

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: n 2 (n/4)2 (n/8)2 (n/4)2 … (n/16)2 (n/2)2 (1) L 2. 15

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: n 2 (n/4)2 (n/8)2 (n/4)2 … (n/16)2 (n/2)2 (1) L 2. 16

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: n 2 (n/4)2 (n/8)2 (n/4)2 … … (n/16)2 (n/2)2 (1) L 2. 17

Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2: n 2 (n/2)2 (n/4)2 (n/8)2 (1) (n/4)2 … … (n/16)2 Total = = (n 2) geometric series L 2. 18

Appendix: geometric series for x 1 for |x| < 1 Return to last slide viewed. L 2. 19

The master method applies to recurrences of the form T(n) = a T(n/b) + f (n) , where a 1, b > 1, and f is asymptotically positive. L 2. 20

Idea of master theorem Recursion tree: f (n) (1) #leaves = ah = alogbn = nlogba a f (n/b) a 2 f (n/b 2) … … a f (n/b) … f (n/b) a h = logbn f (n/b 2) … f (n/b 2) f (n) nlogba (1) L 2. 21

Three common cases Compare f (n) with nlogba: 1. f (n) = O(nlogba – ) for some constant > 0. • f (n) grows polynomially slower than nlogba (by an n factor). Solution: T(n) = (nlogba). L 2. 22

Idea of master theorem Recursion tree: f (n) CASE 1: The weight increases geometrically from the root to the (1) leaves. The leaves hold a constant fraction of the total weight. a f (n/b) a 2 f (n/b 2) … … a f (n/b) … f (n/b) a h = logbn f (n/b 2) … f (n/b 2) f (n) nlogba (1) (nlogba) L 2. 23

Three common cases Compare f (n) with nlogba: 1. f (n) = (nlogba lgkn) for some constant k 0. • f (n) and nlogba grow at similar rates. Solution: T(n) = (nlogba lgk+1 n). L 2. 24

Idea of master theorem Recursion tree: f (n) (1) a f (n/b) a 2 f (n/b 2) … … a f (n/b) … f (n/b) a h = logbn f (n/b 2) … f (n/b 2) f (n) CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels. nlogba (1) (nlogbalg n) L 2. 25

Three common cases (cont. ) Compare f (n) with nlogba: 1. f (n) = (nlogba + ) for some constant > 0. • f (n) grows polynomially faster than nlogba (by an n factor), and f (n) satisfies the regularity condition that a f (n/b) c f (n) for some constant c < 1. Solution: T(n) = ( f (n) ). L 2. 26

Idea of master theorem Recursion tree: f (n) CASE 3: The weight decreases geometrically from the root to the (1) leaves. The root holds a constant fraction of the total weight. a f (n/b) a 2 f (n/b 2) … … a f (n/b) … f (n/b) a h = logbn f (n/b 2) … f (n/b 2) f (n) nlogba (1) ( f (n)) L 2. 27

Examples Ex. T(n) = 4 T(n/2) + n a = 4, b = 2 nlogba = n 2; f (n) = n. CASE 1: f (n) = O(n 2 – ) for = 1. T(n) = (n 2). Ex. T(n) = 4 T(n/2) + n 2 a = 4, b = 2 nlogba = n 2; f (n) = n 2. CASE 2: f (n) = (n 2 lg 0 n), that is, k = 0. T(n) = (n 2 lg n). L 2. 28

Examples Ex. T(n) = 4 T(n/2) + n 3 a = 4, b = 2 nlogba = n 2; f (n) = n 3. CASE 3: f (n) = (n 2 + ) for = 1 and 4(cn/2)3 cn 3 (reg. cond. ) for c = 1/2. T(n) = (n 3). Ex. T(n) = 4 T(n/2) + n 2/lg n a = 4, b = 2 nlogba = n 2; f (n) = n 2/lg n. Master method does not apply. In particular, for every constant > 0, we have n (lg n). L 2. 29

Conclusion • Next time: applying the master method. • For proof of master theorem, goto section L 2. 30