CSE 115ENGR 160 Discrete Mathematics 030112 MingHsuan Yang

CSE 115/ENGR 160 Discrete Mathematics 03/01/12 Ming-Hsuan Yang UC Merced 1

3. 2 Growth of Functions • Study number of operations used by algorithm • For example, given n elements – Study the number of comparisons used by the linear and binary search – Estimate the number of comparisons used by the bubble sort and insertion sort • Use big-O notation to study this irrespective of hardware 2

Big-O notation • Used extensively to estimate the number of operations in algorithm uses as its inputs grows • Determine whether it is practical to use a particular algorithm to solve a problem as the size of the input increases • Can compare with two algorithms and determine which is more efficient 3

Big-O notation • For instance, one algorithm uses 100 n 2+17 n+4 operations and the other uses n 3 operations • Can figure out which is more efficient with big. O notation • The first one is more efficient when n is large even though it uses more operations for smaller values of n, e. g. , n=10 • Related to big-Omega and big-Theta notations in algorithm design 4

Big-O notation • Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers, we say f(x) is O(g(x)) if there are constants C and k such that |f(x)| ≤ C |g(x)| whenever x > k • Read as f(x) is big-oh of g(x) 5

Big-O notation • The constants C and k are called witnesses to the relationship f(x) is O(g(x)) • Need only one pair of witnesses to this relationship • To show f(x) is O(g(x)), need only one pair of constants C and k, s. t. |f(x)| ≤ C |g(x)| • When there is one pair of witnesses, there are infinitely many pairs of witness • If C and k are one pair, any pair C’ and k’ where C<C’ and k<k’, we have |f(x)|≤ C|g(x)|≤ C’|g(x)| when x>k’>k 6

Example • Show that f(x)=x 2+2 x+1 is O(x 2) • We observe that we can estimate size of f(x) when x>1 because x 2>x and x 2>1, 0≤ x 2+2 x+1 ≤ x 2+2 x 2+x 2=4 x 2 when x>1. Thus, we can take C=4 and k=1 to show that f(x) is O(x 2) • Alternatively, when x>2, 2 x≤x 2 and 1≤x 2 0≤ x 2+2 x+1 ≤ x 2+x 2=3 x 2 so C=3, and k=2 are also witnesses to relation f(x) is O(x 2) 7

Example 8

Example • Observe that O(x 2) can be replaced by any function with larger values than x 2, – e. g. , f(x)= x 2+2 x+1 is O(x 3), f(x) is O(x 2+x+7), O(x 2+2 x+1) • In this example, f(x)=x 2+2 x+1, g(x)=x 2, we say both of these big-O relationships are of the same order • Sometimes written as f(x)=O(g(x)) • It is acceptable to write f(x) ∈ O(g(x)) as O(g(x)) represents the set of functions that are O(g(x)) 9

Big-O notation • When f(x) is O(g(x)) and h(x) is a function that has larger absolute values than g(x) does for sufficient large value of x • It follows that f(x) is O(h(x)) |f(x)|≤C|g(x)| if x>k and if |h(x)|>|g(x)| for all x>k, then |f(x)|≤C|h(x)| if x>k • When big-O notation is used, f(x) is O(g(x)) is chosen to be as small as possible 10

Example • Show that 7 x 2 is O(x 3) • When x>7, 7 x 2<x 3, So let C=1 and k=7, we see 7 x 2 is O(x 3) • Alternatively, when x>1, 7 x 2<7 x 3 and so that C=7 and k=1, we have the relationship 7 x 2 is O(x 3) 11

Example • Show that n 2 is not O(n) • To show that n 2 is not O(n), we must show that no pair of constants C and k exist such that n 2≤Cn when n>k • When n>0, we have n≤C to have n 2≤Cn • But no matter what C and k are, the inequality n≤C cannot hold for all n with n>k • In particular, when n is larger than the max of k and C, it is not true n 2≤Cn although n>k 12

Example • Previous example shows that 7 x 2 is O(x 3). Is it also true that x 3 is also O(7 x 2) • To show that, x 3≤ 7 x 2 is equivalent to x≤ 7 C whenever x>k • No C exists for which x≤ 7 C for all x>k • Thus x 3 is not O(7 x 2) 13

Some important big-O results • Let f(x)=anxn+an-1 xn-1+…+a 1 x+a 0, where a 0, a 1, …, an-1, an are all real numbers, then f(x) is O(xn) • Using the triangle inequality, if x>1 14

Example • Big-O notation of the sum of first n positive integers • 1+2+…+n ≤n+n+…+n=n 2, O(n 2) with C=1, k=1 • Alternatively, O(n(n+1)/2)=O(n 2) • Big_O notation of n! • n!=1∙ 2 ∙ 3 ∙ … n ≤n ∙ n … n = nn, O(nn) with C=1 and k=1 • log n!≤log nn=n log n, log n! is O(nlog n) with C=1, k=1 15

Example • We know that n<2 n when n is a positive integer. Show that this implies n is O(2 n) and use this to show that log n is O(n) • n is O(2 n) by taking k=1 and C=1 • Thus, log n<n (base 2), so log n is O(n) • If we have logarithms to a different base b other than 2, we still have logb n is O(n) as logbn = log n/log b < n/log b when n is a positive integer. Take C=1/log b and k=1 16

Growth of functions 17

Growth of combinations of functions • Many algorithms are made up of two or more separate subprocedures • The number of steps is the sum of the number of steps of these subprocedures 18

Theorems • Theorem 2: Suppose f 1(x) is O(g 1(x)) and f 2(x) is O(g 2(x)), – (f 1+f 2)(x) is O(max(|g 1(x)|, |g 2(x)|) – (f 1 f 2)(x) is O(g 1(x)g 2(x)) |(f 1 f 2)(x)|=|f 1(x)||f 2(x)|≤C 1|g 1(x)|C 2|g 2(x)| ≤C 1 C 2|(g 1 g 2)(x)|≤C|(g 1 g 2)(x)| (C= C 1 C 2, k=max(k 1, k 2)) • Corollary: f 1(x) and f 2(x) are both O(g(x)), then (f 1+f 2)(x) is O(g(x)) 19

Example • Big-O notation of f(n)=3 n log(n!)+(n 2+3)logn where n is a positive integer • We know log(n!) is O(nlog n), so the first part is O(n 2 log n) • As n 2+3<2 n 2 when n>2, it follows that n 2+3 is O(n 2), and the second part is O(n 2 log n) • So f(n) is O(n 2 log n) 20

Example • Big-O notation of f(x)=(x+1) log(x 2+1) + 3 x 2 • Note (x+1) is O(x) and x 2+1 ≤ 2 x 2 when x>1 • So, log x 2+1 ≤ log(2 x 2)=log 2+ log x 2=log 2+ 2 log x ≤ 3 log x if x >2 • Thus, log x 2+1 is O(log x) • The first part of f(x) is O(x log x) • Also, 3 x 2 is O(x 2) • So, f(x) is O(max(x log x, x 2))=O(x 2) as x log x ≤ x 2 for x >1 21

Big-Omega • Big-O notation does not provide a lower bound for the size of f(x) for large x • Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say f(x) is ��� (g(x)) if there are positive constants C and k s. t. |f(x)|≥C|g(x)| when x>k • Read as f(x) is big-Omega of g(x) 22

Example • f(x)=8 x 3+5 x 2+7 is �� (g(x)) where g(x)=x 3 • It is easy to see as f(x)= 8 x 3+5 x 2+7≥ 8 x 3 for all positive numbers x • This is equivalent to say g(x)=x 3 is O(8 x 3+5 x 2+7) 23

Big-Theta notation • Want a reference function g(x) s. t. f(x) is O(g(x)) and f(x) is �� (g(x)) • Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say that f(x) is �� (g(x)) if f(x) is O(g(x)) and f(x) is �� (g(x)) • When f(x) is �� (g(x)) , we say f is big-Theta of g(x), and we also say f(x) is of order g(x) 24

Big Theta-notation • When f(x) is �� (g(x)), g(x) is �� (f(x)) • f(x) is �� (g(x)) if and only if f(x) is O(g(x)) and g(x) is O(f(x)) 25

Example • Let f(n)=1+2+…+n. We know that f(n) is O(n 2), to show that f(x) is of order n 2, we need to find C and k s. t. f(n)>Cn 2 for large n • f(n) is O(n 2) and �� (n 2), thus f(n) is �� (n 2) 26

Big-Theta notation • We can show that f(x) is �� (g(x)) if we can find positive real numbers C 1 and C 2 and a positive number k, s. t. C 1|g(x)|≤|f(x)|≤C 2|g(x)| when x>k • This shows f(x) is O(g(x)) and f(x) is �� (g(x)) 27

Example • Show that 3 x 2+8 x log x is �� (x 2) • As 0 ≤ 8 x log x≤ 8 x 2, it follows that 3 x 2+8 x logx ≤ 11 x 2 for x>1 • Consequently 3 x 2+8 x logx is O(x 2) • Clearly 3 x 2+8 x logx is �� (x 2) • Consequently 3 x 2+8 x logx is �� (x 2) 28

Polynomial • One useful fact is that the leading term of a polynomial determines its order • E. g. , f(x)=3 x 5+x 4+17 x 3+2 is of order x 5 • Let f(x)=anxn+an-1 xn-1+…+a 1 x+a 0, where a 0, a 1, …, an-1, an are all real numbers, then f(x) is of order xn 29