CS 575 Design and Analysis of Computer Algorithms
![for loop body 1. 2. for (i=0; i < n; i++) A[i] = i](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-7.jpg "for loop body 1. 2. for (i=0; i < n; i++) A[i] = i")
![for loop body 1. 2. for (i=0; i < n; i++) A[i] = i](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-8.jpg "for loop body 1. 2. for (i=0; i < n; i++) A[i] = i")
![while loop control 1. i = 0 2. while (i < n){ 3. A[i]](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-11.jpg "while loop control 1. i = 0 2. while (i < n){ 3. A[i]")
![while loop control 1. i = 0 2. while (i < n){ 3. A[i]](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-12.jpg "while loop control 1. i = 0 2. while (i < n){ 3. A[i]")
![Example 1 with method 1 1. 2. l for (i=0; i<n; A[i] = i](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-13.jpg "Example 1 with method 1 1. 2. l for (i=0; i<n; A[i] = i")
![Example 1 with method 2 l Method 2 1. 2. for (i=0; i<n; A[i]](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-14.jpg "Example 1 with method 2 l Method 2 1. 2. for (i=0; i<n; A[i]")
![Example 2: What is count 1(2) now? 2. A[i] = i Line 1 i=1](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-15.jpg "Example 2: What is count 1(2) now? 2. A[i] = i Line 1 i=1")
![Example: Finding maximum Findmax(A, n) maximum = A[0]; for (i = 1; i <](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-41.jpg "Example: Finding maximum Findmax(A, n) maximum = A[0]; for (i = 1; i <")
![Termination j=n. l So maximum = max{A[i]|i=0, …, n - 1} l](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-45.jpg "Termination j=n. l So maximum = max{A[i]|i=0, …, n - 1} l")
![Example: Insertion sort INSERTION-SORT(A) 1 for j <- 2 to length[A] 2 key <-](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-48.jpg "Example: Insertion sort INSERTION-SORT(A) 1 for j <- 2 to length[A] 2 key <-")
![Termination j = length[A] + 1. l The array A[1. . length[A]] contains the](https://slidetodoc.com/presentation_image/5b8abd10489186a8e0f68845dc13591a/image-54.jpg "Termination j = length[A] + 1. l The array A[1. . length[A]] contains the")
- Slides: 56
CS 575 Design and Analysis of Computer Algorithms Professor Michal Cutler Lecture 2 Couts
This class Instruction counts l Asymptotic growth functions (Chapter 3) l Proving correctness (17 – 19 2 nd edition) l
Method 1: Sum Line Counts l Derive a count for each line of code taking into account all loop nesting. l Compute total by adding line counts.
Method 2: Barometer Operation l A “barometer instruction” is selected l Count = number of times that barometer instruction is executed. l Search algorithms: – barometer instruction (x == L[j]? ). l Sort algorithms: – barometer instruction (L[i] == L[j]? ). l Should be chosen with care.
Assignment Statement 1. A= B*C-D/F l Count 1 = 1 l In reality? At least 4 Note: When numbers can be very large, algorithms that deal with large numbers will be used and the count will depend also on the number of digits needed to store the large numbers.
Loop condition (i < n)&&(!found) l Count 1 = 1 Note: if loop condition invokes functions count of function must be used 1.
for loop body 1. 2. for (i=0; i < n; i++) A[i] = i Count 2 = 1 Count 1(2) = i=0 i<n false true A[i] = i i=i+1
for loop body 1. 2. for (i=0; i < n; i++) A[i] = i Count 1(2) = i=0 i<n false true A[i] = i =n l last time condition is checked and evaluates to true is when i = n -1 i=i+1
for loop control 1. 2. for (i=0; i < n; i++) <body> Count = number times loop condition is executed (assuming loop condition has a count of 1) i=0 i<n false true Body of the loop i=i+1
for loop control 1. 2. for (i=0; i < n; i++) <body> i=0 Count 1 = number times loop condition i < n is executed i<n = n + 1 Body of the loop Note: last time condition is checked i =n and (n < n) evaluates to false true i=i+1
while loop control 1. i = 0 2. while (i < n){ 3. A[i] = i 4. i = i + 1} Line 1 i=0 Line 2 Count = number times loop condition is executed (assuming loop condition has a count of 1) i<n false true A[i] = i i=i+1 Line 3 Line 4
while loop control 1. i = 0 2. while (i < n){ 3. A[i] = i 4. i = i + 1} Line 2 Count 2 = number times loop condition (i < n) is executed = n + 1 Line 1 i=0 i<n false true A[i] = i i=i+1 Line 3 Line 4
Example 1 with method 1 1. 2. l for (i=0; i<n; A[i] = i Method 1 count 1= n+1 count 1(2) = n*1 = n _________ Total = (n+1)+n = 2 n+1 i++ )
Example 1 with method 2 l Method 2 1. 2. for (i=0; i<n; A[i] = i + 1 i++ ) l Barometer operation = + in body of loop l count 1(+) = n
Example 2: What is count 1(2) now? 2. A[i] = i Line 1 i=1 1. (for i=1; i<=n; i=3*i) Line 1 i <= n no yes A[i] = i Line 2 i = 3* i Line 1
Example 2: What is count 1(2) now? For simplicity, n = 3 k for some positive integer k. Body of the loop executed for i = 1(=30), 31, 32, …, 3 k. So count 1(2)= =k+1 Since k = log 3 n, we get log 3 n + 1
Example 3 Line 1: for (i=0; i<n; i++) Line 2: for (j=0, j<i; j++) Line 3. x = x + 1 Barometer = + Count 1(3(+))=
Time complexity l Practically to get the time complexity: – Select the quickest growing term in the instruction count – If the term is c*f(n) ignore the constant c and consider the asymptotic run time to be O(f(n)) l If the count is the time complexity is O(n 2)
Function Growth The running time of an algorithm as input size approaches infinity is called the asymptotic running time l We study different notations for asymptotic efficiency. l In particular, we study tight bounds, upper bounds and lower bounds. l
Outline for function growth slides Why do we need the different sets? l Definition of the sets Oh, Omega and Theta l Classifying examples: l – Using the original definition – Using limits
The “sets” and their use – Big Oh l l 1. 2. Big “oh” - asymptotic upper bound on the growth of an algorithm When do we use Big Oh? Theory of NP-completeness To provide information on the maximum number of operations that an algorithm performs – Insertion sort is O(n 2) in the worst case • This means that in the worst case it performs at most cn 2 operations – Insertion sort is also O(n 6) in the worst case since it also performs at most dn 6 operations
The “sets” and their use – Omega - asymptotic lower bound on the growth of an algorithm or a problem* When do we use Omega? 1. To provide information on the minimum number of operations that an algorithm performs – Insertion sort is (n) in the best case • This means that in the best case its instruction count is at least cn, – It is (n 2) in the worst case • This means that in the worst case its instruction count is at least dn 2
The “sets” and their use – Omega cont. 2. To provide information on a class of algorithms that solve a problem – Sort algorithms based on comparison of keys are (nlgn) in the worst case • This means that all sort algorithms based only on comparison of keys have to do at least cnlgn operations – Any algorithm based only on comparison of keys to find the maximum of n elements is (n) in every case • This means that all algorithms based only on comparison of keys to find maximum have to do at least cn operations
The “sets” and their use - Theta l Theta - asymptotic tight bound on the growth rate of an algorithm – Insertion sort is (n 2) in the worst and average cases • This means that in the worst and average cases insertion sort performs cn 2 operations – Binary search is (lg n) in the worst and average cases • The means that in the worst case and average cases binary search performs clgn operations
The “sets” and their use – Theta cont. l Note: We want to classify algorithms using Theta. – In Data Structures used Oh l Little “oh” - used to denote an upper bound that is not asymptotically tight. – n is in o(n 3). – n is not in o(n)
The functions Let f(n) and g(n) be asymptotically nonnegative functions whose domains are the set of natural numbers N={0, 1, 2, …}. l A function g(n) is asymptotically nonnegative, if g(n)³ 0 for all n³n 0 where n 0ÎN l
Definition of big Oh O(f(n)) is the set of functions g(n) such that: there exist positive constants c, and n 0 for which, 0£ g(n) £cf(n) for all n³n 0 l f(n) is called an asymptotically upper bound for g(n). l
g(n) Î O(f(n)) t a w o I gr as t s o m sf a t s fa cf(n) g(n) n 0 n
Does 5 n+2 ÎO(n)? Proof: From the definition of Big Oh, there must exist c>0 and integer N>0 such that 0 £ 5 n+2£cn for all n³N. Dividing both sides of the inequality by n>0 we get: 0 £ 5+2/n£c. 2/n £ 2, 2/n>0 becomes smaller when n increases There are many choices here for c and N.
Does 5 n+2 ÎO(n)? If we choose N=1 then 5+2/n£ 5+2/1= 7. So any c ³ 7. Choose c = 7. If we choose c=6, then 0 £ 5+2/n£ 6. So any N ³ 2. Choose N = 2. In either case (we only need one!) we have a c>o and N>0 such that 0 £ 5 n+2£cn for all n ³ N. So the definition is satisfied and 5 n+2 ÎO(n)
Does 2 nÎ O(n)? No. We will prove by contradiction that the definition cannot be satisfied. Assume that n 2Î O(n). From the definition of Big Oh, there must exist c>0 and integer N>0 such that 0 £ n 2£cn for all n³N. Dividing the inequality by n>0 we get 0 £ n £ c for all n³N cannot be true for any n >max{c, N }, contradicting our assumption So there is no constant c>0 such that n£c is satisfied for all n³N, and n 2 O(n)
Definition of the set Omega l (f(n)) is the set of functions g(n) such that: there exist positive constants c, and n 0 for which, 0 £cf(n) £g(n) for all n³n 0 l f(n) is called an asymptotically lower bound for g(n).
g(n) Î (f(n)) at w o r Ig as t s a le sf a t s fa g(n) cf(n) n 0 n
Definition of the set Theta (f(n)) is the set of functions g(n) such that: there exist positive constants c, d, and n 0, for which, 0£ cf(n) £g(n) £df(n) for all n³n 0 l f(n) is called an asymptotically tight bound for g(n). l
g(n) Î (f(n)) df(n) g(n) cf(n) n 0 n row g e W e m a s at rate
Another Definition of Theta log n 5 n + 3 n 1. 5 nlog n n 2 n 5 1/2 n 2 2 n 5 n 2+ 3 n ? ? ?
Limits can be used to determine Order { c then f (n) = ( g (n)) if c > 0 if lim f (n) / g (n) = n®¥ l 0 or c > 0 then f (n) = o ( g(n)) ¥ or c > 0 then f (n) = ( g (n)) The limit must exist
Example using limits
Proving correctness l Proof based on loop invariants – an assertion which is satisfied before each iteration of a loop – At termination the loop invariant provides important property that is used to show correctness l Steps of proof: – Initialization (similar to induction base) – Maintenance (similar to induction proof) – Termination
More on the steps l l l Initialization: Show loop invariant is true before (or at start of) the first execution of a loop Maintenance: Show that if the loop invariant is true before an execution of a loop it is true before the next execution Termination: When the loop terminates, the invariant gives us an important property that helps show the algorithm is correct
Example: Finding maximum Findmax(A, n) maximum = A[0]; for (i = 1; i < n; i++) if A[i] > maximum= A[i] return maximum l What is a loop invariant for this code?
Proof of correctness Loop invariant for Findmax(A): “At the start of the j th iteration (for j = 1, … , n) of the for loop maximum = max{A[i]| i = 0, …, j - 1}” l
Initialization We need to show loop invariant is true at the start of the execution of the for loop l Line 1 sets maximum=A[ 0] = max{A[i]|i=0, …, 1 -1} l l So the loop invariant is satisfied at the start of the for loop.
Maintenance l l l Assume that at the start of the jth iteration of the for loop maximum = max{A[i] | i = 0, …, j - 1} We will show that before the (j + 1)th iteration maximum =max{A[i] | i = 0, …, j} The code computes maximum=max(maximum, A[j]) =max(max{A[i] | i= 0, …, j - 1}, A[j]) = max{A[i] | i = 0, …, j}
Termination j=n. l So maximum = max{A[i]|i=0, …, n - 1} l
Insertion sort What is the main idea? l http: //www. cs. ust. hk/faculty/tcpong/cs 10 2/summer 96/aids/insert. html l
Pseudo code in text 1. 2. 3. Body of a loop indicated by indentation for assignment for i =1 to n is equivalent to for (int i =1; i < (n+1); i++) Arrays are assumed to start at index 1, (left over from old Fortran, Pascal) 4.
Example: Insertion sort INSERTION-SORT(A) 1 for j <- 2 to length[A] 2 key <- A[j] 3 // Insert A[j] into sorted A[1. . j - 1] 4 i <- j – 1 5 while i > 0 and A[i] > key 6 A[i + 1] <- A[i] 7 i <- i – 1 8 A[i + 1] <- key
Proof of correctness l Loop invariant for INSERTIONSORT(A): “At the start of the j th iteration (for j = 2, … , length[A]) of the for loop – A[1. . j -1] contains the elements originally in A[1. . j -1] and – A[1. . j -1] is sorted”
Initialization l l We need to show loop invariant is true at the start of the execution of the for loop After line 1 sets j = 2 and before it compares j to length[A] we have: – Subarray A[1. . 2 - 1]= A[1] contains the original element in A[1] – A[1] is sorted. l So the loop invariant is satisfied
Maintenance If loop invariant is true before this execution of a loop it is true before the next execution l Assume that at the start of the jth iteration of the for loop A[1. . j -1] contains the elements originally in A[1. . j - 1] and A[1. . j -1] is sorted l
Maintenance l We will show that the loop invariant is maintained before the (j + 1)th iteration. – We will show that at the start of the (j + 1)th iteration of the for loop A[1. . j] contains the elements originally in A[1. . j] and A[1. . j] is sorted
Maintenance l l For a more formal proof we need a loop assertion for the while loop We will be less formal and observe that the body of the loop moves A[j - 1], A[j - 2], etc. , to the right until the proper position for A[j] is found and then inserts A[j] into the sub array A[1. . j ]. So: – the sub array A[1. . j] contains the elements originally in A[1. . j] and A[1. . j] is sorted
Termination j = length[A] + 1. l The array A[1. . length[A]] contains the elements originally in A[1. . length[A]] and A[1. . length[A]] is SORTED! l
Loop invariants 1. 2. 3. sum =0; for (i = 0; i < n; i++) sum = sum + A[i]; l What is a loop invariant for this code?
Next class l Recurrences