Algorithm Analysis part 2 CSE 2011 Winter 2011

Algorithm Analysis (part 2) CSE 2011 Winter 2011 9/29/2020 1

Growth Rate l The idea is to establish a relative order among functions for large n. l c , n 0 > 0 such that f(N) c g(N) when N n 0 l f(N) grows no faster than g(N) for “large” N 2

Asymptotic Notation: Big-Oh l f(N) is O(g(N)) if l There are positive constants c and n 0 such that f(N) c. g(N) when N n 0 where c is a real number. l The growth rate of f(N) is less than or equal to the growth rate of g(N). l g(N) is an upper bound on f(N). 3

Big-Oh: Examples l Let f(N) = 2 N 2. Then ¡f(N) is O(N 4) ¡f(N) is O(N 3) ¡f(N) is O(N 2) (best answer, asymptotically tight) l O(N 2): reads “order N-squared” or “Big-Oh Nsquared” 4

Example l Show that 7 N 2 + 10 N + 5 Nlog. N + 3 is O(N 2). l Find c and n 0 such that when N n 0 7 N 2 + 10 N + 5 Nlog. N + 3 c. N 2 l 7 N 2 + 10 N + 5 Nlog. N + 3 7 N 2 + 10 N 2 + 5 N 2 + 3 N 2 25 N 2 when N 1 So c = 25 and n 0 = 1. l Use the same “technique” for the following problems. 5

Big Oh: More Examples l l l N 2 / 2 – 3 N is O(N 2) 1 + 4 N is O(N) 7 N 2 + 10 N + 3 is O(N 2), is also O(N 3) log 10 N = log 2 N / log 2 10 is O(log 2 N) or O(log N) sin N is O(1); 10 is O(1), 1010 is O(1) l l log N + N is O(N) logk N is O(N) for any constant k N is O(2 N), but 2 N is not O(N) 210 N is not O(2 N) 6

Math Review: Logarithmic Functions 7

Some Rules When considering the growth rate of a function using O() l Ignore the lower order terms and the coefficients of the highest-order term l No need to specify the base of logarithm ¡Changing the base from one constant to another changes the value of the logarithm by only a constant factor l If T 1(N) is O(f(N) and T 2(N) is O(g(N)), then ¡T 1(N) + T 2(N) is O(f(N) + g(N)) (or less formally it is max (O(f(N)), O(g(N)))), ¡T 1(N) * T 2(N) is O(f(N) * g(N)) 8

Big-Omega l c , n 0 > 0 such that f(N) c g(N) when N n 0 l f(N) grows no slower than g(N) for “large” N 9

Big-Omega l f(N) is (g(N)) if l There are positive constants c and n 0 such that f(N) c g(N) when N n 0 where c is a real number. l The growth rate of f(N) is greater than or equal to the growth rate of g(N). l g(N) is a lower bound on f(N). 10

Big-Omega: Examples l Let f(N) = 2 N 2. Then ¡f(N) is (N) (not tight) ¡f(N) is (N 2) (best answer) 11

Big-Theta l The growth rate of f(N) is the same as the growth rate of g(N) l f(N) is (g(N)) iff f(N) is O(g(N)) and f(N) is (g(N)) 12

Big-Theta: Example l Let f(N) = N 2 , g(N) = 2 N 2 ¡Since f(N) is O(g(N)) and f(N) is (g(N)), f(N) = (g(N)). l c 1 = 1, n 1 = 0 l c 2 = ½, n 2 = 0 13

Typical Growth Rates

Some More Rules l If T(N) is a polynomial of degree k, then T(N) is (Nk). l For logarithmic functions, T(logm N) is (log N). l logk N is O(N) for any constant k (logarithms grow very slowly) 15

Small-oh l f(N) is o(g(N)) if l c, n 0 such that f(N) < c g(N) when N > n 0 l Less formally, f(N) is o(g(N)) if f(N) is O(g(N)) and f(N) is not (g(N)) l g(N) grows faster than f(N) for “large” N. 16

Small-oh: Example l Let f(N) = ¾ N 2 and f(N) be o(g(N)). ¡g(N) = N 2 ? ¡g(N) = N 2 log N ? ¡g(N) = N 3 ? 17

Determining Relative Growth Rates of Two Functions 1. Using simple algebra (slide 14) Example: which function grows faster? ¡f(N) = N log. N ¡g(N) = N 1. 5 2. Using L’Hôpital’s rule 18

Using L' Hôpital's Rule l L' Hôpital's rule If and then = l Determine the relative growth rates: compute ¡ if 0: f(N) is o(g(N)) ¡ if constant 0: f(N) is (g(N)) ¡ if : g(N) is o(f(N)) ¡ limit oscillates: no relation 19

Summary of Chapter 4 l Given an algorithm, compute its running time in terms of O, , and (if any). ¡ Usually the big-Oh running time is enough. l Given f(n) = 5 n + 10, show that f(n) is O(n). ¡ Find c and n 0 l Compare the grow rates of 2 functions. l Order the grow rates of several functions. ¡ Use slide 14. ¡ Use L’Hôpital’s rule.

Next time … l Recursion (Chapter 3) l Reading: Chapter 4 21