The Lower Bounds of Problems 2012116 1 Lower

The Lower Bounds of Problems 2012/11/6 1

Lower bound n Def : A lower bound of a problem is the least time complexity required for any algorithm which can be used to solve this problem. ☆ worst case lower bound ☆ average case lower bound 2

Measure the difficulty of a problem n n We can use the lower bound to measure the difficulty of a problem. With the lower bound, we can answer the questions: n Is the algorithm for a problem the best (optimal)? 3

Lower bound n The lower bound for a problem is not unique. n n n At present, if the highest lower bound of a problem is (n log n) and the time complexity of the best algorithm is O(n 2). n n e. g. (1), (n log n) are all lower bounds for sorting. ( (1), (n) are trivial) We may try to find a higher lower bound. We may try to find a better algorithm. Both of the lower bound and the algorithm may be improved. If the present lower bound is (n log n) for a problem and there is an algorithm with time complexity O(n log n), then the algorithm is optimal. 4

Asymptotic Notations n n Def: f(n) = O(g(n)) “upper bound" iff c, n 0 |f(n)| c|g(n)| n n 0 e. g. f(n) = 3 n 2 + 2 g(n) = n 2 n 0=2, c=4 f(n) = O(n 2) e. g. f(n) = n 3 + n = O(n 3) e. g. f(n) = 3 n 2 + 2 = O(n 3) or O(n 100 )

asymptotic upper bound

Def : f(n) = (g(n)) “lower bound” iff c, and n 0, |f(n)| c|g(n)| n n 0 n n e. g. f(n) = 3 n 2 + 2 = (n 2) or (n)

asymptotic lower bound

n n Def : f(n) = (g(n)) iff c 1, c 2, and n 0, c 1|g(n)| |f(n)| c 2|g(n)| n n 0 e. g. f(n) = 3 n 2 + 2 = (n 2)

n f(n) = (g(n))

Theorem n For any two functions f(n) and g(n), if and only if and.

The worst case lower bound of sorting 6 permutations for 3 data elements a 1 a 2 a 3 1 2 3 1 3 2 2 1 3 2 3 1 2 3 2 1 12

straight insertion sort: n n input data: (2, 3, 1) (1) a 1: a 2 (2) a 2: a 3, a 2 a 3 (3) a 1: a 2, a 1 a 2 input data: (2, 1, 3) (1)a 1: a 2, a 1 a 2 (2)a 2: a 3 13

Decision tree for straight insertion sort: 14

Decision tree for bubble sort: 15

Worst case lower bound of sorting n n To find the worst case lower bound, we have to find the smallest depth of a binary tree. n! distinct permutations: n! leaf nodes in the binary decision tree. balanced tree has the smallest depth: log(n!) = (n log n) Worst case lower bound for sorting: (n log n) 16

log n!=O(n log n) n Stirling approximation: n n! n log n! log n log n (n log n) 17

Average case lower bound of sorting n n n A sorting algorithm corresponds to a binary decision tree. The average time complexity of a sorting algorithm: the external path length of the binary tree n! The external path length is minimized if the tree is balanced (all leaf nodes on level d or level d 1) 18

Average case lower bound of sorting 19

Compute the min external path length 1. Depth of balanced binary tree with c leaf nodes: d = log c Leaf nodes can appear only on level d or d 1. 2. x 1 leaf nodes on level d 1 x 2 leaf nodes on level d x 1 + x 2 = c x 1 + = 2 d-1 x 1 = 2 d c x 2 = 2(c 2 d-1) 20

Average case lower bound of sorting 3. External path length: M= x 1(d 1) + x 2 d = (2 d c)(d 1) + (2 c 2 d)d = c + cd 2 d 4. d = log c d log c + 1 c + cd 2 d c + c(log c) -2 2 log c = c log c – c. Thus M > c log c – c = n! log n! – n! M/n! > log n! -1 = (n log n) Average case lower bound of sorting: (n log n) 21

External Path Length of a tree n The sum of the lengths of paths from the root to each of the leaf nodes. 22

Finding lower bound by problem transformation n n The lower bound of Problem A is known. Problem A reduces to problem B (A B) iff A can be solved by using any algorithm which solves B. n n Let ALGB be the algorithm with the lowest time complexity to solve the problem B, and let ALGA be the algorithm solving the problem A on the basis of ALG B. So, L(A) T(ALGA) T(tr 1) + T(tr 2) + T(ALGB). If holds L(A) T(ALGA) T(tr 1) + T(tr 2) + T(ALGB) = T(ALGB) if T(tr 1) + T(tr 2) T(ALGA) If A B, B is more difficult. So, the lower bound of A is also the lower bound of B. 23

The lower bound of the convex hull problem n n sorting convex hull A B an instance of A: (x 1, x 2, …, xn) ↓transformation an instance of B: {( x 1, x 12), ( x 2, x 22), …, ( xn, xn 2)} assume: x 1 < x 2 < …< xn n E. G. A: (2, 8, 3, 5) B: {(2, 4), (8, 64), (3, 9), (5, 25)} The answer of B: {(2, 4), (3, 9), (5, 25), (8, 64)} The answer of A: 2, 3, 5, 8 24

The lower bound of the convex hull problem n If the convex hull problem can be solved, we can also solve the sorting problem. n n The lower bound of sorting: (n log n) Thus, the lower bound of the convex hull problem: (n log n) 25