Orders of Growth Introduction 2 2 Growth of

Orders of Growth – Introduction 2. 2 Growth of Functions For functions over numbers, we often need to know a rough measure of how fast a function grows. (과연 함수는 얼마나 빨리 증가하는가? ) If f(x) is faster growing than g(x), then f(x) always eventually becomes larger than g(x) in the limit (for large enough values of x). (f(x)가 g(x)보다 빨리 증가한다면, 궁극적으로 f(x)의 값이 g(x)의 값보다 커진다. ) Useful in engineering for showing that one design scales better or worse than another. (공학에서는 어느 것이 다른 것보다 얼마나 나쁜지/좋은지를 보여주는 척도(의 정도)가 필요하다…) 2 Discrete Mathematics by Yang-Sae Moon

Orders of Growth – Motivation 2. 2 Growth of Functions Suppose you are designing a web site to process user data. Suppose database • program A takes f. A(n)=30 n+8 microseconds to process any n records, while • program B takes f. B(n)=n 2+1 microseconds to process the n records. Which program do you choose, knowing you’ll want to support millions of users? 3 Discrete Mathematics by Yang-Sae Moon

On a graph, as you go to the right, a faster growing function eventually becomes larger. . . Value of function Visualizing Orders of Growth 2. 2 Growth of Functions f. A(n)=30 n+8 f. B(n)=n 2+1 Increasing n 4 Discrete Mathematics by Yang-Sae Moon

Concept of Orders of Growth 2. 2 Growth of Functions We say f. A(n)=30 n+8 is order n, or O(n). It is, at most, roughly proportional to n. f. B(n)=n 2+1 is order n 2, or O(n 2). It is roughly proportional to n 2. Any O(n 2) function is faster-growing than any O(n) function. For large numbers of user records, the O(n 2) function will always take more time. 5 Discrete Mathematics by Yang-Sae Moon

Definition: O(g), at most order g 2. 2 Growth of Functions Let g be any function R R. Define “at most order g”, written O(g), to be: {f: R R | c, k: x>k: f(x) cg(x)}. • “Beyond some point k, function f is at most a constant c times g (i. e. , proportional to g). ” • (x > k일 때, f(x) cg(x)를 만족하는 c, k가 존재하면, f는 O(g)라 한다. ) “f is at most order g”, or “f is O(g)”, or “f=O(g)” all just mean that f O(g). Sometimes the phrase “at most” is omitted. (일반적으로 그냥 “오더 g”라 이야기한다. ) 6 Discrete Mathematics by Yang-Sae Moon

Points about the Definition 2. 2 Growth of Functions Note that f is O(g) so long as any values of c and k exist that satisfy the definition. But: The particular c, k, values that make the statement true are not unique: Any larger value of c and/or k will also work. (조건을 만족하는 c, k는 여러 개 있을 수 있다. ) You are not required to find the smallest c and k values that work. (Indeed, in some cases, there may be no smallest values!) (조건을 만족하는 가장 작은 c, k를 찾을 필요는 없다. 보여주기만 하면 된다. ) However, you should prove that the values you choose do work. (증명하기 위해서는 만족하는 c, k를 찾아야 한다. ) 7 Discrete Mathematics by Yang-Sae Moon

“Big-O” Proof Example 2. 2 Growth of Functions Show that 30 n+8 is O(n). • Show c, k: n>k: 30 n+8 cn. • Let c=31, k=8. Assume n>k(=8). • Then cn = 31 n = 30 n + n > 30 n+8, so 30 n+8 < cn. Show that n 2+1 is O(n 2). • Show c, k: n>k: n 2+1 cn 2. • Let c=2, k=1. Assume n>1. • Then cn 2 = 2 n 2 = n 2+n 2 > n 2+1, or n 2+1< cn 2. 8 Discrete Mathematics by Yang-Sae Moon

Note 30 n+8 isn’t less than n anywhere (n>0). It isn’t even less than 31 n everywhere. But it is less than 31 n everywhere to the right of n=8. Value of function Big-O Example, Graphically 2. 2 Growth of Functions cn = 31 n 30 n+8 O(n) n>k=8 Increasing n 9 Discrete Mathematics by Yang-Sae Moon

Examples of Big-O (1/2) 2. 2 Growth of Functions f(x) = anxn + an-1 xn-1 + … + a 1 x + a 0 • f(x) = anxn + an-1 xn-1 + … + a 1 x + a 0 |an|xn + |an-1|xn-1 + … + |a 1|x + |a 0| = xn(|an| + |an-1|/x + … + |a 1|/xn-1 + |a 0|/xn) xn(|an| + |an-1| + … + |a 1| + |a 0|) = cxn = O(xn) c f(n) = 1 + 2 + 3 + … + n • f(n) = 1 + 2 + 3 + … + n n+n+n+…+n = n 2 = O(n 2) 10 Discrete Mathematics by Yang-Sae Moon

Examples of Big-O (2/2) 2. 2 Growth of Functions f(n) = n! • f(n) = n! = n∙(n-1)∙(n-2)∙…∙ 3∙ 2∙ 1 n∙n∙n∙…∙n∙n∙n = nn = O(xn) f(n) = log(n!) • f(n) = log(n!) log(nn) = nlogn = O(nlogn) 11 Discrete Mathematics by Yang-Sae Moon

Useful Facts about Big-O (1/2) 2. 2 Growth of Functions Big O, as a relation, is transitive: f O(g) g O(h) f O(h) Sums of functions: If g O(f) and h O(f), then g+h O(f). c>0, O(cf)=O(f+c)=O(f) • E. g. , O(2 x 2) = O(x 2+2) = O(x 2 -3) = O(x 2) If f 1=O(g 1) and f 2 = O(g 2), then f 1+f 2 = O(max(g 1, g 2)) • E. g. , if f 1(x) = x 2 = O(x 2), f 2(x) = x 3 = O(x 3), then (f 1+f 2)(x) = x 2 + x 3 = O(x 3) = O(max(x 2, x 3)) 12 Discrete Mathematics by Yang-Sae Moon

Useful Facts about Big-O (2/2) 2. 2 Growth of Functions f 1=O(g 1) and f 2 = O(g 2), then f 1 f 2 = O(g 1 g 2) • E. g. , if f 1(x) = x 2 = O(x 2), f 2(x) = x 3 = O(x 3), then (f 1 f 2)(x) = x 2∙x 3 = x 5 = O(x 3) = O(x 2∙x 3)) Other useful facts… • f, g & constants a, b R, with b 0, - af = O(f) - f+O(f) = O(f) (e. g. 3 x 2 = O(x 2)) (e. g. x 2+x = O(x 2)) • Also, if f= (1) (at least order 1), then: - |f|1 -b = O(f); (logb |f|)a = O(f) g=O(fg) fg O(g) a=O(f) (e. g. x 1 = O(x)) log x = O(x)) x = O(x log x)) x log x O(x)) 3 = O(x)) 13 Discrete Mathematics by Yang-Sae Moon

Definition: (g), exactly order g 2. 2 Growth of Functions If f O(g) and g O(f) then we say “g and f are of the same order” or “f is (exactly) order g” and write f (g). (f(x)=O(g(x))이고 g(x)=O(f(x))이면, f(x)= (g(x))이며 g(x)= (f(x))이다. ) Another equivalent definition: (g) {f: R R | c 1 c 2 k x>k: |c 1 g(x)| |f(x)| |c 2 g(x)| } (c 1 g(x) f(x) c 2 g(x)를 만족하는 c 1과 c 2가 존재하면 f(x)= (g(x))이다. ) One more definition: (g) = {f | g O(f)} (i. e. , g(x)=O(f(x)) “The functions that are at least order g. ” (하한 정의) 14 Discrete Mathematics by Yang-Sae Moon

Rules for 2. 2 Growth of Functions Mostly like rules for O( ), except: f, g>0 & constants a, b R, af (f), but f (fg) unless g= (1) |f|1 -b (f), and (logb|f|)c (f). with b>0, Same as with O. Unlike with O. 15 Discrete Mathematics by Yang-Sae Moon

Examples of (1/2) 2. 2 Growth of Functions f(n) = 1 + 2 + 3 + … + n = (n 2)? • f(n) = 1 + 2 + 3 + … + n n+n+n+…+n = n 2 • f(n) = 1 + 2 + 3 + … + n = (n∙(n – 1))/2 = n 2/2 – n/2 n 2/2 • n 2/2 f(n) n 2, i. e. , c 1 = ½, c 2 = 1 f(n) = (n 2) 16 Discrete Mathematics by Yang-Sae Moon

Examples of (2/2) 2. 2 Growth of Functions f(y) = 3 y 2 + 8 ylogy = (y 2)? • f(n) = 3 y 2 + 8 ylogy 11 y 2 (if y > 1) (since 8 ylogy 8 y 2) • f(n) = 3 y 2 + 8 ylogy y 2 (if y > 1) • y 2 f(y) 11 y 2, i. e. , c 1 = 1, c 2 = 11 f(y) = (y 2) 17 Discrete Mathematics by Yang-Sae Moon