Discrete Mathematics CS 2610 February 26 2009 part

Discrete Mathematics CS 2610 February 26, 2009 -- part 1

Big-O Notation Big-O notation is used to express the time complexity of an algorithm n n We can assume that any operation requires the same amount of time. The time complexity of an algorithm can be described independently of the software and hardware used to implement the algorithm. 2

Big-O Notation Def. : Let f , g be functions with domain R 0 or N and codomain R. f(x) is O(g(x)) if there are constants C and k st x > k, |f (x )| C |g (x )| f (x ) is asymptotically dominated by g (x ) C|g(x)| is an upper bound of f(x). C and k are called witnesses to the relationship between f & g. C|g(x)| |f(x)| k 3

Big-O Notation To prove that a function f(x) is O(g(x)) n Find values for k and C, not necessarily the smallest one, larger values also work!! n It is sufficient to find a certain k and C that works n In many cases, for all x 0, if f(x) 0 then |f(x)| = f(x) Example: f(x) = x 2 + 2 x + 1 is O(x 2) for C = 4 and k = 1 4

Big-O Notation Show that f(x) = x 2 + 2 x + 1 is O(x 2). When x > 1 we know that x ≤ x 2 and 1 ≤ x 2 then 0 ≤ x 2 + 2 x + 1 ≤ x 2 + 2 x 2 + x 2 = 4 x 2 so, let C = 4 and k = 1 as witnesses, i. e. , f(x) = x 2 + 2 x + 1 < 4 x 2 when x > 1 Could try x > 2. Then we have 2 x ≤ x 2 & 1 ≤ x 2 then 0 ≤ x 2 + 2 x + 1 ≤ x 2 + x 2 = 3 x 2 so, C = 3 and k = 2 are also witnesses to f(x) being O(x 2). Note that f(x) is also O(x 3), etc. 5

Big-O Notation Show that f(x) = 7 x 2 is O(x 3). When x > 7 we know that 7 x 2 < x 3 (multiply x > 7 by x 2) so, let C = 1 and k = 7 as witnesses. Could try x > 1. Then we have 7 x 2 < 7 x 3 so, C = 7 and k = 1 are also witnesses to f(x) being O(x 3). Note that f(x) is also O(x 4), etc. 6

Big-O Notation Show that f(n) = n 2 is not O(n). Show that no pair of C and k exists such that n 2 ≤ Cn whenever n > k. When n > 0, divide both sides of n 2 ≤ Cn by n to get n ≤ C. No matter what C and k are, n ≤ C will not hold for all n with n > k. 7

Big-O Notation Observe that g(x) = x 2 is O(x 2 + 2 x + 1) Def: Two functions f(x) and g(x) have the same order iff g(x) is O(f(x)) and f(x) is O(g(x)) 8

Big-O Notation Also, the function f(x) = 3 x 2 + 2 x + 3 is O(x 3) What about O(x 4) ? In fact, the function Cg(x) is an upper bound for f(x), but not necessarily the tightest bound. n When Big-O notation is used, g(x) is chosen to be as small as possible. 9

Big-Oh - Theorem: If f(x) = anxn+ an-1 xn-1+…+ a 1 x+ a 0 where ai R, i=0, …n; then f(x) is O(xn). Leading term dominates! Proof: if x > 1 we have |f(x)| = |anxn+ an-1 xn-1+…+ a 1 x+ a 0| ≤ |an|xn+ |an-1|xn-1+…+ |a 1|x+ |a 0| = xn(|an| + |an-1|/x +…+ |a 1|/xn-1 + |a 0|/xn) ≤ xn(|an| + |an-1| +…+ |a 1| + |a 0|) So, |f(x)| ≤ Cxn where C = |an| + |an-1| +…+ |a 1| + |a 0| whenever x > 1 (what’s k? k = 1, why? ) What’s this: |a + b| ≤ |a| + |b| 10

Big-O Example: Prove that f(n) = n! is O(nn) Proof (easy): n! = 1 · 2 · 3 · 4 · 5 · · · n ≤n·n·n···n = nn where our witnesses are C = 1 and k = 1 Example: Prove that log(n!) is O(nlogn) Using the above, take the log of both sides: log(n!) ≤ log(nn) which is equal to n log(n) 11

Big-O Lemma: A constant function is O(1). Proof: Left to the viewer The most common functions used to estimate the time complexity of an algorithm. (in increasing O() order): 1, (log n), n, (n log n), n 2, n 3, … 2 n, n! 12

Big-O Properties Transitivity: if f is O(g) and g is O(h) then f is O(h) Sum Rule: n If f 1 is O(g 1) and f 2 is O(g 2) then f 1+f 2 is O(max(|g 1|, |g 2|)) n If f 1 is O(g) and f 2 is O(g) then f 1+f 2 is O(g) Product Rule n If f 1 is O(g 1) andf 2 is O(g 2) then f 1 f 2 is O(g 1 g 2) For all c > 0, O(cf), O(f + c), O(f c) are O(f) 13

Big-O – Properties Example: Give a big-O estimate for 3 n log (n!) + (n 2+3)log n, n>0 1) For 3 n log (n!) we know log(n!) is O(nlogn) and 3 n is O(n) so we know 3 n log(n!) is O(n 2 logn) 2) For (n 2+3)log n we have (n 2+3) < 2 n 2 when n > 2 so it’s O(n 2); and (n 2+3)log n is O(n 2 log n) 3) Finally we have an estimate for 3 n log (n!) + (n 2+3)log n that is: O(n 2 log n) 14

Big-O Notation Def. : Functions f and g are incomparable, if f(x) is not O(g) and g is not O(f). f: R+ R, f(x) = 5 x 1. 5 g: R+ R, g(x) = |x 2 sin x| -- 5 x 1. 5 -- |x 2 sin x| -- x 2 15

Big-Omega Notation Def. : Let f, g be functions with domain R 0 or N and codomain R. f(x) is (g(x)) if there are positive constants C and k such that x > k, C |g (x )| |f (x )| v C |g(x)| is a lower bound for |f(x)| C|g(x)| k 16

Big-Omega Property Theorem: f(x) is (g(x)) iff g(x) is O(f(x)). Is this trivial or what? 17

Big-Omega Property Example: prove that f(x) = 3 x 2 + 2 x + 3 is (g(x)) where g(x) = x 2 Proof: first note that 3 x 2 + 2 x + 3 ≥ 3 x 2 for all x ≥ 0. That’s the same as saying that g(x) = x 2 is O(3 x 2 + 2 x + 3) 18

Big-Theta Notation Def. : Let f , g be functions with domain R 0 or N and codomain R. f(x) is (g(x)) if f(x) is O(g(x)) and f(x) is (g(x)). C 2|g(x)| |f(x)| C 1|g(x)| 19

Big-Theta Notation When f(x) is (g(x)), we know that g(x) is (f(x)). Also, f(x) is (g(x)) iff f(x) is O(g(x)) and g(x) is O(f(x)). Typical g functions: xn, cx, log x, etc. 20

Big-Theta Notation To prove that f(x) is order g(x) n n Method 1 w Prove that f is O(g(x)) w Prove that f is (g(x)) Method 2 w Prove that f is O(g(x)) w Prove that g is O(f(x)) 21

Big-Theta Example show that 3 x 2 + 8 x log x is (x 2) (or order x 2) 0 ≤ 8 x log x ≤ 8 x 2 so 3 x 2 + 8 x log x ≤ 11 x 2 for x > 1. So, 3 x 2 + 8 x log x is O(x 2) (can I get a witness? ) Is x 2 O(3 x 2 + 8 x log x)? You betcha! Why? Therefore, 3 x 2 + 8 x log x is (x 2) 22

Big Summary Upper Bound – Use Big-Oh Lower Bound – Use Big-Omega Upper and Lower (or Order of Growth) – Use Big-Theta 23

Time to Shift Gears Again Number Theory Livin’ Large 24