CPSC 311 Analysis of Algorithms Sorting Lower Bound

CPSC 311 Analysis of Algorithms Sorting Lower Bound Prof. Jennifer Welch Fall 2009 CPSC 411, Fall 2008: Set 2 1

Comparison Based Sorting n All these algorithms are comparison based n n the behavior depends on relative values of keys, not exact values behavior on [1, 3, 2, 4] is same as on [9, 25, 23, 99] Fastest of these algorithms was O(nlog n). n We will show that's the best you can get with comparison based sorting. n CPSC 411, Fall 2008: Set 2 2

Decision Tree n n n Consider any comparison based sorting algorithm Represent its behavior on all inputs of a fixed size with a decision tree Each tree node corresponds to the execution of a comparison Each tree node has two children, depending on whether the parent comparison was true or false Each leaf represents correct sorted order for that path CPSC 411, Fall 2008: Set 2 3

Decision Tree Diagram first comparison: check if ai ≤ aj YES NO second comparison if ai ≤ aj : check if ak ≤ a l YES NO third comparison if ai ≤ aj and ak ≤ al : check if ax ≤ ay second comparison if ai > aj : check if am ≤ ap YES NO CPSC 411, Fall 2008: Set 2 4

Insertion Sort for j : = 2 to n to key : = a[j] i : = j-1 while i > 0 and a[i] > key do a[i]+1 : = a[i] i : = i -1 endwhile a[i+1] : = key endfor CPSC 411, Fall 2008: Set 2 5

Insertion Sort for n = 3 a 1 ≤ a 2 ? YES NO a 2 ≤ a 3 ? YES a 1 ≤ a 3 ? NO a 1 a 2 a 3 NO YES a 1 ≤ a 3 ? a 2 a 1 a 3 YES NO a 1 a 3 a 2 a 3 a 1 a 2 ≤ a 3 ? YES NO a 2 a 3 a 1 CPSC 411, Fall 2008: Set 2 a 3 a 2 a 1 6

Insertion Sort for n = 3 a 1 ≤ a 2 ? YES NO a 2 ≤ a 3 ? YES a 1 ≤ a 3 ? NO a 1 a 2 a 3 NO YES a 1 ≤ a 3 ? a 2 a 1 a 3 YES NO a 1 a 3 a 2 a 3 a 1 a 2 ≤ a 3 ? YES NO a 2 a 3 a 1 CPSC 411, Fall 2008: Set 2 a 3 a 2 a 1 7

How Many Leaves? n Must be at least one leaf for each permutation of the input n n n otherwise there would be a situation that was not correctly sorted Number of permutations of n keys is n!. Idea: since there must be a lot of leaves, but each decision tree node only has two children, tree cannot be too shallow n depth of tree is a lower bound on running time CPSC 411, Fall 2008: Set 2 8

Key Lemma Height of a binary tree with n! leaves is (n log n). Proof: Maximum number of leaves in a binary tree with height h is 2 h. h = 1, 21 leaves h = 2, 22 leaves h = 3, 23 leaves CPSC 411, Fall 2008: Set 2 9

Proof of Lemma n n Let h be height of decision tree. Number of leaves in decision tree, which is n!, is at most 2 h. 2 h ≥ n! h ≥ log(n!) = log(n(n-1)…(2)(1)) ≥ (n/2)log(n/2) by algebra = (n log n) CPSC 411, Fall 2008: Set 2 10

Finishing Up Any binary tree with n! leaves has height (n log n). n Decision tree for any c-b sorting alg on n keys has height (n log n). n Any c-b sorting alg has at least one execution with (n log n) comparisons n Any c-b sorting alg has (n log n) worstcase running time. n CPSC 411, Fall 2008: Set 2 11