CS 221 Algorithms and Data Structures Lecture 3

CS 221: Algorithms and Data Structures Lecture #3. 5 Sorting Takes Priority Steve Wolfman 2014 W 1 1

Today’s Outline • Sorting with Priority Queues, Three Ways 2

Quick Review of Sorts • Insertion Sort: Keep a list of already sorted elements. One by one, insert new elements into the right place in the sorted list. • Selection Sort: Repeatedly find the smallest (or largest) element and put it in the next slot (where it belongs in the final sorted list). • Merge Sort: Divide the list in half, sort the halves, merge them back together. (Base case: length 1. )3

Insertion Sort Invariant in sorted order untouched each iteration moves the line right one element 4

Selection Sort Invariant smallest, 2 nd smallest, 3 rd smallest, … in order remainder in no particular order each iteration moves the line right one element 5

How Do We Sort with a Priority Queue? You have a bunch of data. You want to sort by priority. You have a priority queue. WHAT DO YOU DO? G(9) insert F(7) E(5) D(100) A(4) B(6) delete. Min C(3) 6

“PQSort” (Super-Ridiculously Vague Pseudocode) Sort(elts): pq = new PQ for each elt in elts: pq. insert(elt); // or all at once if that’s easier sorted. Elts = new array of size elts. length for i = 0 to elts. length – 1: sorted. Elts[i] = pq. delete. Min // or all at once return sorted. Elts What sorting algorithm is this? a. b. c. d. e. Insertion Sort Selection Sort Heap Sort Merge Sort None of these 7

Reminder: Naïve Priority Q Data Structures • Unsorted array (or linked list): – insert: worst case O(1) – delete. Min: worst case O(n) • Sorted array: – insert: worst case O(n) – delete. Min: worst case O(1) 8

“PQSort” delete. Maxes with Unsorted List MAX-PQ 0 1 2 3 4 5 5 9 4 8 1 6 6 7 8 10 12 13 9 10 11 12 13 2 3 14 20 7 How long does inserting all of these elements take? And then the deletions…

“PQSort” delete. Maxes with Unsorted List MAX-PQ 0 1 2 3 4 5 5 9 4 8 1 6 6 7 8 10 12 13 PQ 9 10 11 12 13 2 3 14 20 7

“PQSort” delete. Maxes with Unsorted List MAX-PQ 0 1 2 3 4 5 5 9 4 8 1 6 6 7 8 10 12 13 9 10 11 12 13 2 3 14 20 7 2 3 14 20 PQ 5 9 4 8 1 6 10 12 13 PQ 7 Result

“PQSort” delete. Maxes with Unsorted List MAX-PQ 0 1 2 3 4 5 5 9 4 8 1 6 6 7 8 10 12 13 9 10 11 12 13 2 3 14 20 7 2 3 14 20 PQ 5 9 4 8 1 6 10 12 13 PQ 7 Result 2 3 7 14 20 Result

“PQSort” delete. Maxes with Unsorted List MAX-PQ 0 1 2 3 4 5 5 9 4 8 1 6 6 7 8 10 12 13 9 10 11 12 13 2 3 14 20 7 2 3 14 20 PQ 5 9 4 8 1 6 10 12 13 PQ 5 9 4 8 1 6 Result 10 12 13 2 3 PQ 5 9 4 8 1 6 10 12 PQ 7 7 14 20 Result 7 2 3 13 14 20 13 Result

Two PQSort Tricks 1) Use the array to store both your results and your PQ. No extra memory needed! 2) Use a max-heap to sort in increasing order (or a min-heap to sort in decreasing order) so your heap doesn’t “move” during deletions. 14

“PQSort” delete. Maxes with Unsorted List MAX-PQ How long does “build” take? No time at all! How long do the deletions take? Worst case: O(n 2) What algorithm is this? a. Insertion Sort b. Selection Sort c. Heap Sort d. Merge Sort e. None of these 5 9 4 8 1 6 10 12 PQ 7 2 3 13 14 20 15 Result

“PQSort” insertions with Sorted List MAX-PQ 0 1 2 3 4 5 5 9 4 8 1 6 6 7 8 10 12 13 9 10 11 12 13 2 3 14 20 7 PQ 5 5 5 9 9 9 4 4 4 8 8 8 1 1 1 6 6 6 10 12 13 2 2 2 3 3 3 14 7 20 7 PQ 14 20 16 PQ

“PQSort” insertions with Sorted List MAX-PQ How long does “build” take? Worst case: O(n 2) How long do the deletions take? What algorithm is this? a. Insertion Sort b. Selection Sort c. Heap Sort d. Merge Sort e. None of these 5 9 4 8 1 6 10 12 13 2 3 7 14 20 17 PQ

“PQSort” Build with Heap MAX-PQ 0 1 2 3 4 5 5 9 4 8 1 6 6 7 8 10 12 13 9 10 11 12 13 2 3 14 20 7 2 1 4 7 Floyd’s Algorithm 20 13 14 12 3 6 10 9 8 5 PQ Takes only O(n) time! 18

“PQSort” delete. Maxes with Heap MAX-PQ 0 1 2 3 20 13 14 12 4 5 6 7 8 9 10 11 12 13 3 6 10 9 8 2 1 4 5 7 8 2 1 4 5 20 8 2 1 4 14 20 PQ 5 4 2 1 13 14 20 PQ 14 13 10 12 3 6 7 9 PQ 13 12 10 12 9 10 9 8 3 3 6 6 7 7 5 PQ 19 Totally incomprehensible as an array!

“PQSort” delete. Maxes with Heap MAX-PQ 5 9 4 8 1 6 10 12 13 2 3 14 20 7 5 9 4 8 12 13 1 2 6 3 14 20 7 10 20

“PQSort” delete. Maxes with Heap MAX-PQ 5 20 9 8 12 13 2 4 1 13 6 3 14 20 10 7 14 12 9 3 8 2 6 1 4 10 5 7 Build Heap 21 Note: 9 ends up being perc’d down as well since its invariant is violated by the time we reach it.

“PQSort” delete. Maxes with Heap MAX-PQ 20 13 14 12 9 3 8 2 13 6 1 4 12 10 5 9 7 3 8 2 2 4 6 1 4 9 7 5 10 20 5 3 8 2 7 6 10 12 13 14 20 7 8 5 3 4 2 7 14 20 4 12 9 1 6 1 10 8 3 12 10 9 5 13 14 6 9 1 12 13 14 20 10 8 5 3 4 2 6 1 7 13 14 20

“PQSort” with Heap MAX-PQ How long does “build” take? Worst case: O(n) How long do the deletions take? Worst case: O(n lg n) 9 What algorithm is this? a. Insertion Sort 8 7 b. Selection Sort 5 3 6 1 c. Heap Sort 2 4 10 12 13 14 20 d. Merge Sort e. None of these 9 8 7 5 3 6 PQ 1 2 4 10 12 13 14 20 Result 23

“PQSort” What sorting algorithm is this? a. Insertion Sort b. Selection Sort c. Heap Sort d. Merge Sort e. None of these Sort(elts): pq = new PQ for each elt in elts: pq. insert(elt); sorted. Elts = new array of size elts. length for i = 0 to elements. length – 1: sorted. Elts[i] = pq. delete. Min return sorted. Elts 24

To Do • Read: Epp Section 9. 5 and KW Section 10. 1, 10. 4, and 10. 7 -10. 10 25

Coming Up • More sorting! 26