Onotation upper bound Asymptotic running times of algorithms

O-notation (upper bound) • Asymptotic running times of algorithms are usually defined by functions whose domain are N={0, 1, 2, …} (natural numbers) Formal Definition of O-notation f(n) = O(g(n)) if positive constants c, n 0 such that 0 ≤ f(n) ≤ cg(n), n ≥ n 0 Ex. 2 n 2 = O(n 3) 2 n 2 ≤ cn 3 cn ≥ 2 c = 1 & n 0 = 2 or c = 2 & n 0 = 1 2 n 3 = O(n 3) 2 n 3 ≤ cn 3 c ≥ 2 c = 2 & n 0 = 1 Analysis of Algorithms 1

O-notation (upper bound) • “=” is funny; “one-way” equality • O-notation is sloppy, but convenient. – We must understand what O(n) really means • O(g(n)) is actually a set of functions. O(g(n)) = { f(n) : positive constants c, n 0 such that 0 ≤ f(n) ≤ cg(n), n ≥ n 0 } 2 n 2 = O(n 3) means that 2 n 2 O(n 3) Analysis of Algorithms 2

O-notation (upper bound) • O-notation is an upper-bound notation • It makes no sense to say “running time of an algorithm is at least O(n 2)”. • let running time be T(n) ≥ O(n 2) means T(n) ≥ h(n) for some h(n) O(n 2) however, this is true for any T(n) since h(n) = 0 O(n 2), & running time > 0, so stmt tells nothing about running time Analysis of Algorithms 3

O-notation (upper bound) Analysis of Algorithms 4

-notation (lower bound) Formal Definition of -notation f(n) = (g(n)) if positive constants c, n 0 such that 0 ≤ cg(n) ≤ f(n) , n ≥ n 0 (g(n)) { f(n) : positive constants c, n 0 such that 0 ≤ cg(n) ≤ f(n) , n ≥ n 0 } Analysis of Algorithms 5

-notation (lower bound) Analysis of Algorithms 6

Θ-notation (tight bound) Formal Definition of Θ-notation f(n) = Θ(g(n)) if positive constants c 1, c 2, n 0 such that 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n) , n ≥ n 0 Θ(g(n)) { f(n) : positive constants c 1, c 2, n 0 such that 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n) , n ≥ n 0 } Analysis of Algorithms 7

Θ-notation (tight bound) - Example Analysis of Algorithms 8

Θ-notation (tight bound) - 0 < c 1 ≤ h(n) ≤ c 2 Analysis of Algorithms 9

Θ-notation (tight bound) - 0 < c 1 ≤ h(n) ≤ c 2 Analysis of Algorithms 10

Θ-notation (tight bound) Θ(g(n)) { f(n) : positive constants c 1, c 2, n 0 such that 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n) , n ≥ n 0 } Analysis of Algorithms 11

Θ-notation Analysis of Algorithms 12

Θ-notation Analysis of Algorithms 13

Using O-notation for Describing Running Times • O-notation is used to bound worst-case running times – bounds running time on arbitrary inputs as well • O(n 2) bound on worst-case running time of insertion sort also applies to its running time on every input • What we really mean by “running time of insertion sort is O(n 2)” – worst-case running time of insertion sort is O(n 2) or equivalently – no matter what particular input of size n is chosen (for each value of) running time on that set of inputs is O(n 2) Analysis of Algorithms 14

Using Ω-notation for Describing Running Times • Ω(n) is used to bound the best-case running times – bounds the running time on arbitrary inputs as well • e. g. , Ω(n) bound on best-case running time of insertion sort – running time of insertion sort is Ω(n) • “running time of an algorithm is Ω(g(n))” means – no matter what particular input of size n is chosen (for any n), running time on that set of inputs is at least a constant times g(n), for sufficiently large n – however, it is not contradictory to say “worst-case running time of insertion sort is Ω(n 2)” since there exists an input that causes algorithm to take Ω(n 2) time Analysis of Algorithms 15

Using Θ-notation for Describing Running Times Case 1. used to bound worst-case & best-case running times of an algorithm if they are not asymptotically equal Case 2. used to bound running time of an algorithm if its worst & best case running times are asymptotically equal Analysis of Algorithms 16

Using Θ-notation for Describing Running Times Case (1) • a Θ-bound on worst-/best-case running time does not apply to its running time on arbitrary inputs • e. g. , Θ(n 2) bound on worst-case running time of insertion sort does not imply a Θ(n 2) bound on running time of insertion sort on every input since T(n) = O(n 2) & T(n) = Ω(n) for insertion sort Analysis of Algorithms 17

Using Θ-notation for Describing Running Times Case (2) • implies a Θ-bound on every input e. g. , merge sort T(n) = O(nlgn) T(n) = Ω(nlgn) T(n) = Θ(nlgn) Analysis of Algorithms 18

Asymptotic Notation In Equations Analysis of Algorithms 19

Asymptotic notation appears on LHS of an equation Analysis of Algorithms 20

Other asymptotic notations – o-notation (small o) Analysis of Algorithms 21

o-notation Analysis of Algorithms 22

ω-notation (small omega notation) Analysis of Algorithms 23

Asymptotic comparison of functions Analysis of Algorithms 24

Analogy to the comparison of two real numbers Analysis of Algorithms 25

Analogy to the comparison of two real numbers Analysis of Algorithms 26