Introduction to Algorithms Recurrences Solving Recurrences Recurrences are

  • Slides: 34
Download presentation
Introduction to Algorithms: Recurrences

Introduction to Algorithms: Recurrences

Solving Recurrences • Recurrences are like solving integrals, differential equations, etc. • Three Common

Solving Recurrences • Recurrences are like solving integrals, differential equations, etc. • Three Common Approaches: • Recursion Tree • Substitution • Master Theorem • Next Lecture: Applications of recurrences to divide-and-conquer algorithms. CS 421 - Analysis of Algorithms 2

Recursion-Tree Method • A recursion tree models the costs (time) of a recursive execution

Recursion-Tree Method • A recursion tree models the costs (time) of a recursive execution of an algorithm. • The recursion-tree method can be unreliable, just like any method that uses ellipses (…). • The recursion-tree method promotes intuition, however. • The recursion tree method is good for generating guesses for the substitution method. CS 421 - Analysis of Algorithms 3

Example of recursion tree Solve: T(n) = T(n/4) + T(n/2) + n 2

Example of recursion tree Solve: T(n) = T(n/4) + T(n/2) + n 2

Example of Recursion Tree T(n) = T(n/4) + T(n/2) + n 2: T(n) CS

Example of Recursion Tree T(n) = T(n/4) + T(n/2) + n 2: T(n) CS 421 - Analysis of Algorithms 5

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2 CS 421 - Analysis of Algorithms 6

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2 T(n/16) T(n/8) T(n/4) CS 421 - Analysis of Algorithms 7

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2 (1) CS 421 - Analysis of Algorithms 8

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2 (1) CS 421 - Analysis of Algorithms 9

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2 5 n 2 16 (1) CS 421 - Analysis of Algorithms 10

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2 … 5 n 2 16 25 n 2 256 (1) CS 421 - Analysis of Algorithms 11

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2 5 n 2 16 25 n 2 256 (1) CS 421 - Analysis of Algorithms 12

Geometric Series CS 421 - Analysis of Algorithms 13

Geometric Series CS 421 - Analysis of Algorithms 13

Geometric Series : Positive Fractions CS 421 - Analysis of Algorithms 14

Geometric Series : Positive Fractions CS 421 - Analysis of Algorithms 14

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2

Example of recursion tree T(n) = T(n/4) + T(n/2) + n 2: n 2 5 n 2 16 25 n 2 256 (1) CS 421 - Analysis of Algorithms 15

Substitution method The most general method: 1. Guess the form of the solution. 2.

Substitution method The most general method: 1. Guess the form of the solution. 2. Verify by induction. 3. Solve for constants. CS 421 - Analysis of Algorithms 16

Substitution method EXAMPLE: T(n) = 4 T(n/2) + n Make assumption that T(1) =

Substitution method EXAMPLE: T(n) = 4 T(n/2) + n Make assumption that T(1) = (1). 1. Guess O(n 3). • Prove O and separately. • Assume that T(k) ck 3 for k < n. 2. Prove T(n) cn 3 by induction. 3. Solve for any constants. CS 421 - Analysis of Algorithms 17

Example of substitution T (n) 4 T (n / 2) n 4 c(n /

Example of substitution T (n) 4 T (n / 2) n 4 c(n / 2)3 n (c / 2)n 3 n cn 3 ((c / 2)n 3 n) cn 3 desired – residual CS 421 - Analysis of Algorithms 18

Example (continued) • We must also handle the initial conditions, i. e. , ground

Example (continued) • We must also handle the initial conditions, i. e. , ground the induction with base cases. • Base: T(n) = (1) for all n < n 0, where n 0 is a suitable constant. • For 1 n < n 0, we have “ (1)” cn 3, if we pick c big enough. CS 421 - Analysis of Algorithms 19

Example (continued) • We must also handle the initial conditions, i. e. , ground

Example (continued) • We must also handle the initial conditions, i. e. , ground the induction with base cases. • Base: T(n) = (1) for all n < n 0, where n 0 is a suitable constant. • For 1 n < n 0, we have “ (1)” cn 3, if we pick c big enough. This bound is not tight! CS 421 - Analysis of Algorithms 20

A Tighter Upper Bound Prove that T(n) = O(n 2).

A Tighter Upper Bound Prove that T(n) = O(n 2).

A Tighter Upper Bound We shall prove that T(n) = O(n 2). Assume that

A Tighter Upper Bound We shall prove that T(n) = O(n 2). Assume that T(k) ck 2 for k < n T (n) 4 T (n / 2) n 4 c(n / 2)2 n cn 2 n O(n 2 ) CS 421 - Analysis of Algorithms 22

A Tighter Upper Bound We shall prove that T(n) = O(n 2). Assume that

A Tighter Upper Bound We shall prove that T(n) = O(n 2). Assume that T(k) ck 2 for k < n T (n) 4 T (n / 2) n 4 c(n / 2)2 n cn 2 n O(n 2 ) Wrong! We must prove the I. H. CS 421 - Analysis of Algorithms 23

A Tighter Upper Bound We shall prove that T(n) = O(n 2). Assume that

A Tighter Upper Bound We shall prove that T(n) = O(n 2). Assume that T(k) ck 2 for k < n T (n) 4 T (n / 2) n 4 c(n / 2)2 n cn 2 n O(n 2 ) cn 2 ( n) [ desired – residual ] cn 2 CS 421 - Analysis of Algorithms 24

A Tighter Upper Bound We shall prove that T(n) = O(n 2). Assume that

A Tighter Upper Bound We shall prove that T(n) = O(n 2). Assume that T(k) ck 2 for k < n T (n) 4 T (n / 2) n 4 c(n / 2)2 n cn 2 n O(n 2 ) cn 2 ( n) [ desired – residual ] cn 2 for no choice of c > 0. Lose! CS 421 - Analysis of Algorithms 25

A Tighter Upper Bound! IDEA: Strengthen the inductive hypothesis. Subtract a low-order term. Inductive

A Tighter Upper Bound! IDEA: Strengthen the inductive hypothesis. Subtract a low-order term. Inductive hypothesis: T(k) c 1 k 2 – c 2 k for k < n. CS 421 - Analysis of Algorithms 26

A Tighter Upper Bound! T(k) c 1 k 2 – c 2 k for

A Tighter Upper Bound! T(k) c 1 k 2 – c 2 k for k < n: T(n) = 4 T(n/2) + n = 4(c 1(n/2)2 – c 2(n/2)) + n = c 1 n 2 – 2 c 2 n + n = c 1 n 2 – c 2 n – (c 2 n – n) c 1 n 2 – c 2 n, if c 2 1. CS 421 - Analysis of Algorithms 27

A Tighter Upper Bound! T(k) c 1 k 2 – c 2 k for

A Tighter Upper Bound! T(k) c 1 k 2 – c 2 k for k < n: T(n) = 4 T(n/2) + n = 4(c 1(n/2)2 – c 2(n/2)) + n = c 1 n 2 – 2 c 2 n + n = c 1 n 2 – c 2 n – (c 2 n – n) c 1 n 2 – c 2 n, if c 2 1. Pick c 1 big enough to handle the initial conditions. CS 421 - Analysis of Algorithms 28

The Master Method CS 421 - Analysis of Algorithms 29

The Master Method CS 421 - Analysis of Algorithms 29

Three Common Cases CS 421 - Analysis of Algorithms 30

Three Common Cases CS 421 - Analysis of Algorithms 30

Examples : Master Method CS 421 - Analysis of Algorithms 31

Examples : Master Method CS 421 - Analysis of Algorithms 31

Examples : Master Method CS 421 - Analysis of Algorithms 32

Examples : Master Method CS 421 - Analysis of Algorithms 32

Examples : Master Method CS 421 - Analysis of Algorithms 33

Examples : Master Method CS 421 - Analysis of Algorithms 33

Examples : Master Method CS 421 - Analysis of Algorithms 34

Examples : Master Method CS 421 - Analysis of Algorithms 34

Algorithms Recurrences CS 8833 Algorithms Recurrences Definition aAlgorithms Recurrences CS 8833 Algorithms Recurrences Definition a
Solving Recurrences Thomas Schwarz SJ Recurrences The equivalentSolving Recurrences Thomas Schwarz SJ Recurrences The equivalent
Solving recurrences Recurrences are a major tool forSolving recurrences Recurrences are a major tool for
Analysis of Algorithms Recurrences Recurrences and Running TimeAnalysis of Algorithms Recurrences Recurrences and Running Time
Recurrences It continues Jeff Chastine Recurrences When anRecurrences It continues Jeff Chastine Recurrences When an
CS 332 Algorithms Solving Recurrences Continued The MasterCS 332 Algorithms Solving Recurrences Continued The Master
CS 332 Algorithms Merge Sort Solving Recurrences TheCS 332 Algorithms Merge Sort Solving Recurrences The
Analysis of Algorithms Solving Recurrences Continued The MasterAnalysis of Algorithms Solving Recurrences Continued The Master
CIS 313 Algorithms Merge Sort Solving Recurrences TheCIS 313 Algorithms Merge Sort Solving Recurrences The
Introduction to Algorithms BruteForce Algorithms Introduction to AlgorithmsIntroduction to Algorithms BruteForce Algorithms Introduction to Algorithms
Introduction to Algorithms DividenConquer Algorithms Introduction to AlgorithmsIntroduction to Algorithms DividenConquer Algorithms Introduction to Algorithms
Solving Recurrences Continued The Master Theorem Introduction toSolving Recurrences Continued The Master Theorem Introduction to
Introduction to Algorithms Greedy Algorithms Greedy Algorithms AIntroduction to Algorithms Greedy Algorithms Greedy Algorithms A
CS 2133 Data Structures Merge Sort Solving RecurrencesCS 2133 Data Structures Merge Sort Solving Recurrences
Lecture 9 Divide and Conquer Solving Recurrences CpLecture 9 Divide and Conquer Solving Recurrences Cp
Merge Sort Solving Recurrences The Master Theorem DavidMerge Sort Solving Recurrences The Master Theorem David
Practice in Algorithm Analysis 1 Solving Recurrences WritePractice in Algorithm Analysis 1 Solving Recurrences Write
Merge Sort Solving Recurrences The Master Theorem DavidMerge Sort Solving Recurrences The Master Theorem David
Lecture 8 Solving Recurrences CSE 373 Data StructuresLecture 8 Solving Recurrences CSE 373 Data Structures
Analysis of Algorithms CS 477677 Recurrences Instructor GeorgeAnalysis of Algorithms CS 477677 Recurrences Instructor George
Analysis of Algorithms Chapter 02 Recurrences 1 MethodsAnalysis of Algorithms Chapter 02 Recurrences 1 Methods
CSE 421 Algorithms Richard Anderson Lecture 11 RecurrencesCSE 421 Algorithms Richard Anderson Lecture 11 Recurrences
Recurrences Methods and Examples CSE 2320 Algorithms andRecurrences Methods and Examples CSE 2320 Algorithms and
Analysis of Algorithms Chapter 02 Recurrences 1 MethodsAnalysis of Algorithms Chapter 02 Recurrences 1 Methods