 # CS 6045 Advanced Algorithms Sorting Algorithms Sorting Input

• Slides: 31 CS 6045: Advanced Algorithms Sorting Algorithms Sorting • Input: sequence of numbers Output: a sorted sequence Insertion Sort Insertion Sort //next current //go left //find place for current // shift sorted right // go left //put current in place Example Correctness • Which elements are in sorted order after running each iteration? • Loop invariant: the subarray A[1 … j-1] consists of the elements originally in A[1 … j-1] but in sorted order Correctness • To use a loop invariant to prove correctness, we must show three things about it: – Initialization: It is true prior to the first iteration of the loop. – Maintenance: If it is true before an iteration of the loop, it remains true before the next iteration. – Termination: When the loop terminates, the invariant—usually along with the reason that the loop terminated—gives us a useful property that helps show that the algorithm is correct. Insertion Sort Correctness • Initialization: Just before the first iteration, j = 2. The subarray A[1 … j-1] is the single element A, which is the element originally in A, and it is trivially sorted. • Maintenance: To be precise, we would need to state and prove a loop invariant for the “inner” while loop. Rather than getting bogged down in another loop invariant, we instead note that the body of the inner while loop works by moving A[1 … j-1], A[1 … j-2], A[1 … j-3], and so on, by one position to the right until the proper position for key (which has the value that started out in A[j]) is found. At that point, the value of key is placed into this position. • Termination: The outer for loop ends when j > n, which occurs when j = n+1. Therefore, j – 1 = n. Plugging n in for j - 1 in the loop invariant, the subarray A[1 … n] consists of the elements originally in A[1 … n] but in sorted order. In other words, the entire array is sorted . Analyze Algorithm’s Running Time • Depends on – input size – input quality (partially ordered) • Kinds of analysis – Worst case (standard) – Average case (sometimes) – Best case (never) Asymptotic Analysis • Ignore machine dependent constants • Look at growth of T(n) while n – Drop lower-order terms – Ignore the constant coefficient in the leading term • O - big O notation to represent the order of growth Asymptotic Notations • BIG O: O – f = O(g) if f is no faster then g – f / g < some constant • BIG OMEGA: – f = (g) if f is no slower then g – f / g > some constant • BIG Theta: – f = (g) if f has the same growth rate as g – some constant < f / g < some constant Analyze Insertion Sort Insertion Sort Analysis • Best Case – O(n) • Worst Case – O(n^2) • Average Case – O(n^2) Merge Sort • • Divide (into two equal parts) Conquer (solve for each part separately) Combine separate solutions Merge sort – Divide into two equal parts – Sort each part using merge-sort (recursion!!!) – Merge two sorted subsequences Merge Sort Example 1 Example 2 Merging • Design an algorithm, which takes O(n) time?    Analyze Merge Sort 12345678 1 358 15 5 2467 38 1 8 log n 47 3 7 26 4 • n comparisons per level • log n levels • total runtime = n log n 6 2 Quicksort • Sorts in place like insertion unlike merge • Divide into two parts such that – elements of left part < elements of right part • Conquer: recursively solve for each part separately • Combine: trivial - do not do anything Quicksort(A, p, r) if p <r then q Partition(A, p, r) Quicksort(A, p, q-1) Quicksort(A, q+1, r) //divide //conquer left //conquer right Divide = Partition PARTITION(A, p, r) //Partition array from A[p] to A[r] with pivot A[r] //Result: All elements original A[r] has index i x = A[r] i =p-1 for j = p to r - 1 if A[j] <= x i=i+1 exchange A[i] A[j] exchange A[i+1] with A[r] return i + 1  Loop Invariant Runtime of Quicksort • Worst case: – Partition cause one sub-problem with n-1 elements and one with 0 elements – O(n^2) 0123456789 0 123456789 n 89 8 9 Runtime of Quicksort • Best case: – every time partition in (almost) equal parts – O(n log n) • Average case – O(n log n) Randomized Quicksort • Idea: select a randomly chosen element as the pivot • Randomized algorithms: – includes (pseudo) random-number generator – the behavior depends not only from the input but from random-number generator also • Simple approach: permute randomly the input – same result but more difficult to analyze Randomized Quicksort Randomized Quicksort • Partition around first element: O(n^2) worst -case • Average case: O(n log n)