MS 101 Algorithms Instructor Neelima Gupta nguptacs du

MS 101: Algorithms Instructor Neelima Gupta ngupta@cs. du. ac. in

Table Of Contents Solving Recurrences The Master Theorem

Recurrence Relations

Recurrences • The expression: is a recurrence. – Recurrence: an equation that describes a function in terms of its value on smaller functions

Recurrence Examples

Solving Recurrences • Substitution method • Iteration method • Master method

Solving Recurrences • The substitution method (CLR 4. 1) – “Making a good guess” method – Guess the form of the answer, then use induction to find the constants and show that solution works – Examples: • T(n) = 2 T(n/2) + (n) T(n) = (n lg n) • T(n) = 2 T( n/2 ) + n ? ? ?

Solving Recurrences • The substitution method (CLR 4. 1) – “Making a good guess” method – Guess the form of the answer, then use induction to find the constants and show that solution works – Examples: • T(n) = 2 T(n/2) + (n) T(n) = (n lg n) • T(n) = 2 T( n/2 ) + n T(n) = (n lg n) • T(n) = 2 T( n/2 )+ 17) + n ? ? ?

Substitution method • Guess the form of the solution. • Use mathematical induction to find the constants and show that the solution works. The substitution method can be used to establish either upper or lower bounds on a recurrence.

An example (Substitution method ) • T(n) = 2 T(floor(n/2) ) +n We guess that the solution is T(n)=0(n lg n). i. e. to show that T(n) ≤ cn lg n , for some constant c> 0 and n ≥ m. Assume that this bound holds for [n/2]. So , we get T(n) ≤ 2(c floor (n/2) lg(floor(n/2))) + n ≤ cn lg(n/2) + n = cn lg n – cn lg 2 + n = cn lg n – cn + n ≤ cn lg n where , the last step holds as long as c≥ 1.

• Boundary conditions : Suppose , T(1)=1 is the sole boundary condition of the recurrence. then , for n=1 , the bound T(n)≤ c n lg n yields T(1)≤ c lg 1=0 , which is at odds with T(1)=1. Thus , the base case of our inductive proof fails to hold. To overcome this difficulty , we can take advantage of the asymptotic notation which only requires us to prove T(n)≤c n lg n for n≥ m. The idea is to remove the difficult boundary condition T(1)= 1 from consideration. Thus , we can replace T(1) by T(2) as the base cases in the inductive proof , letting m=2.

Contd. . From the recurrence , with T(1) = 1, we get T(2)=4 We require T(2)≤ c 2 lg 2 It is clear that , any choice of c≥ 2 suffices for the base cases

Are we done? • Have we proved the case for n =3. • Have we proved that T(3) ≤ c 3 lg 3. • No. Since floor(3/2) = 1 and for n =1, it does not hold. So induction does not apply on n= 3. • From the recurrence, with T(1) = 1, we get T(3) = 5. The inductive proof that T(n) ≤ c n lg n for some constant c≥ 1 can now be completed by choosing c large enough that T(3)≤c 3 lg 3 also holds. It is clear that , any choice of c ≥ 2 is sufficient for this to hold. Thus we can conclude that T(n) ≤ c n lg n for any c ≥ 2 and n ≥ 2.

What if we have extra lower order terms?

Wrong Application of induction

Solving Recurrences • Another option is “iteration method” – Expand the recurrence – Work some algebra to express as a summation – Evaluate the summation • We will show some examples

Assignment 4 • Solve the following recurrence: T(n) = T(αn) + T(βn) + where 0 < α ≤ β < 1 n, Assume suitable initial conditions.

The Master Theorem • Given: a divide and conquer algorithm – An algorithm that divides the problem of size n into a subproblems, each of size n/b – Let the cost of each stage (i. e. , the work to divide the problem + combine solved subproblems) be described by the function f(n) • Then, the Master Theorem gives us a cookbook for the algorithm’s running time:

The Master Theorem • if T(n) = a. T(n/b) + f(n) then

Using The Master Method • T(n) = 9 T(n/3) + n – a=9, b=3, f(n) = n – nlog a = nlog 9 = (n 2) – Since f(n) = O(nlog 9 - ), where =1, case 1 applies: b 3 3 – Thus the solution is T(n) = (n 2)

More Examples of Master’s Theorem • • • T(n) = 3 T(n/5) + n T(n) = 2 T(n/2) + 1 T(n) = T(n/2) + n T(n) = T(n/2) + 1

When Master’s Theorem cannot be applied • T(n) = 2 T(n/2) + n logn • T(n) = 2 T(n/2) + n/ logn

Up Next Proving the correctness of Algorithms