CS 575 Design and Analysis of Computer Algorithms

CS 575 Design and Analysis of Computer Algorithms Professor Michal Cutler Lecture 3 Asymptotic growth functions

This class Growth functions l Proof of correctness l

Questions Meaning of three sets l Usages of three sets l Asymptotic nonnegative functions l

Definition of big Oh O(f(n)) is the set of functions g(n) such that: there exist positive constants c, and n 0 for which, 0£ g(n) £cf(n) for all n³n 0 l f(n) is called an asymptotically upper bound for g(n). l

g(n) O(f(n)) t a w o I gr as t s o m sf a t s fa cf(n) g(n) n 0 n

Definition of the set Omega l W(f(n)) is the set of functions g(n) such that: there exist positive constants c, and n 0 for which, 0 £cf(n) £g(n) for all n³n 0 l f(n) is called an asymptotically lower bound for g(n).

g(n) W(f(n)) at w o r Ig as t s a le sf a t s fa g(n) cf(n) n 0 n

Definition of the set Theta Q(f(n)) is the set of functions g(n) such that: there exist positive constants c, d, and n 0, for which, 0£ cf(n) £g(n) £df(n) for all n³n 0 l f(n) is called an asymptotically tight bound for g(n). l

g(n) Q(f(n)) df(n) g(n) cf(n) n 0 n row g e W e m a s at rate

Another Definition of Theta log n 5 n + 3 n 1. 5 nlog n n 2 n 5 1/2 n 2 2 n 5 n 2+ 3 n ? ? ?

Limits can be used to determine Order { c then f (n) = Q ( g (n)) if c > 0 if lim f (n) / g (n) = n®¥ l 0 or c > 0 then f (n) = o ( g(n)) ¥ or c > 0 then f (n) = W ( g (n)) The limit must exist

Are they true? l 1, 000 n 2 O(n 2) why/why not? l (n - 1)n / 2 O(n 2) why /why not? l lg (n 2) O( lg n ) why /why not? l n 2 O(n) why /why not?

Is 5 n-20 W (n)? Proof: From the definition of Omega, there must exist c>0 and integer N>0 such that 0 £ cn £ 5 n-20 for all n³N Dividing the inequality by n>0 we get: 0 £ c £ 5 -20/n for all n³N. 20/n £ 20, and 20/n becomes smaller as n grows. There are many choices here for c and N. Since c > 0, 5 – 20/n >0 and N >4 If we choose N=5, then 1 £ 5 -20/5 £ 5 -20/n. So 0 < c £ 1. Choose c = ½. If we choose c=4, then 5 – 20/n ³ 4 and N ³ 20. Choose N = 20. In either case (we only need one!) we have a c>o and N>0 such that 0 £ cn £ 5 n-20 for all n ³ N. So the definition is satisfied and 5 n-20 W (n)

Are they true? l 1, 000 n 2 W (n 2) why /why not? l (n - 1)n / 2 W (n 2) why /why not? l lg (n 2) W ( lg n ) why /why not? l n 2 W (n) why /why not?

More Q l 1, 000 n 2 Q(n 2) why /why not? l (n - 1)n / 2 Q(n 2) why /why not? l lg (n 2) Q ( lg n ) why /why not? l n 2 Q(n) why /why not?

L’Hopital’s Rule If f(x) and g(x) are both differentiable with derivatives f’(x) and g’(x), respectively, and if

Example using limits

Example using limit

Example using limits

Comparing ln n with na a > 0 l Using limits and L’hopital we get: l So ln n = O(na) for any a > 0 When the exponent a is very small, we need to look at very large values of n to see that na > ln n l

Values for log 10 n and n. 01

Values for log 10 n and n. 001

Proving correctness l Proof based on loop invariants – an assertion which is satisfied before each iteration of a loop – At termination the loop invariant provides important property that is used to show correctness l Steps of proof: – Initialization (similar to induction base) – Maintenance (similar to induction proof) – Termination

More on the steps l l l Initialization: Show loop invariant is true before (or at start of) the first execution of a loop Maintenance: Show that if the loop invariant is true before an execution of a loop it is true before the next execution Termination: When the loop terminates, the invariant gives us an important property that helps show the algorithm is correct

Example: Finding maximum Findmax(A, n) maximum = A[0]; for (i = 1; i < n; i++) if A[i] > maximum= A[i] return maximum l What is a loop invariant for this code?

Proof of correctness Loop invariant for Findmax(A): “At the start of the j th iteration (for j = 1, … , n) of the for loop maximum = max{A[i]| i = 0, …, j - 1}” l

Initialization We need to show loop invariant is true at the start of the execution of the for loop l Line 1 sets maximum=A[ 0] = max{A[i]|i=0, …, 1 -1} l l So the loop invariant is satisfied at the start of the for loop.

Maintenance l l l Assume that at the start of the jth iteration of the for loop maximum = max{A[i] | i = 0, …, j - 1} We will show that before the (j + 1)th iteration maximum =max{A[i] | i = 0, …, j} The code computes maximum=max(maximum, A[j]) =max(max{A[i] | i= 0, …, j - 1}, A[j]) = max{A[i] | i = 0, …, j}

Termination j=n. l So maximum = max{A[i]|i=0, …, n - 1} l

Insertion sort What is the main idea? l http: //www. cs. ust. hk/faculty/tcpong/cs 10 2/summer 96/aids/insert. html l

Pseudo code in text 1. 2. 3. Body of a loop indicated by indentation for assignment for i =1 to n is equivalent to for (int i =1; i < (n+1); i++) Arrays are assumed to start at index 1, (left over from old Fortran, Pascal) 4.

Example: Insertion sort INSERTION-SORT(A) 1 for j <- 2 to length[A] 2 key <- A[j] 3 // Insert A[j] into sorted A[1. . j - 1] 4 i <- j – 1 5 while i > 0 and A[i] > key 6 A[i + 1] <- A[i] 7 i <- i – 1 8 A[i + 1] <- key

Proof of correctness l Loop invariant for INSERTIONSORT(A): “At the start of the j th iteration (for j = 2, … , length[A]) of the for loop – A[1. . j -1] contains the elements originally in A[1. . j -1] and – A[1. . j -1] is sorted”

Initialization l l We need to show loop invariant is true at the start of the execution of the for loop After line 1 sets j = 2 and before it compares j to length[A] we have: – Subarray A[1. . 2 - 1]= A[1] contains the original element in A[1] – A[1] is sorted. l So the loop invariant is satisfied

Maintenance If loop invariant is true before this execution of a loop it is true before the next execution l Assume that at the start of the jth iteration of the for loop A[1. . j -1] contains the elements originally in A[1. . j - 1] and A[1. . j -1] is sorted l

Maintenance l We will show that the loop invariant is maintained before the (j + 1)th iteration. – We will show that at the start of the (j + 1)th iteration of the for loop A[1. . j] contains the elements originally in A[1. . j] and A[1. . j] is sorted

Maintenance l l For a more formal proof we need a loop assertion for the while loop We will be less formal and observe that the body of the loop moves A[j - 1], A[j - 2], etc. , to the right until the proper position for A[j] is found and then inserts A[j] into the sub array A[1. . j ]. So: – the sub array A[1. . j] contains the elements originally in A[1. . j] and A[1. . j] is sorted

Termination j = length[A] + 1. l The array A[1. . length[A]] contains the elements originally in A[1. . length[A]] and A[1. . length[A]] is SORTED! l

Example: Sum 1. 2. 3. sum =0; for (i = 0; i < n; i++) sum = sum + A[i]; l What is a loop invariant for this code?

Proof of correctness l Loop invariant: “At the start of the j th iteration (for j = 0, … , n) of the for loop

Initialization l Show loop invariant is true at the start of the execution of the for loop l Line 1 sets sum = 0+ l So the loop invariant is satisfied at the start of iteration 0.

Maintenance If loop invariant is true before this execution of a loop it is true before the next execution l Assume that at the start of the jth iteration of the for loop l

Maintenance l We will show that at the start of the (j + 1)th iteration of the for loop l From the code

Termination j=n l So l

Next class l Solving recurrences