Heap Heap Sort Priority Queues Binary heap as

Heap, Heap Sort, Priority Queues - Binary heap as an application of binary trees Sorting with a data structure support Designing and analyzing the priority queue data structure

- Similar to a queue/stack, but with “priorities” PRIORITY QUEUES 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 1

Priority Queues • Stack: FILO – push(), pop(), top() • Queue: FIFO – push_front(), pop_back() • Double Ended Queue: FIFO & LIFO – push_front(), back(), pop_front(), pop_back() • Priority Queue: – Each element has a “priority” – push() stores new element (with “priority”) – pop() removes element with highest/lowest priority – top() shows that element but doesn’t remove 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 2

A Priority Queue in Picture 3 10/31/2020 Push() CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 3

A Priority Queue in Picture 7 Push() 3 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 4

A Priority Queue in Picture 5 Push() 3 7 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 5

A Priority Queue in Picture 3 5 top() 7 7 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 6

A Priority Queue in Picture 3 5 pop() 7 7 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 7

A Priority Queue in Picture 2 push() 3 5 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 8

A Priority Queue in Picture 3 5 2 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 9

Where is a Priority Queue used? • In all kinds of networking protocols – Buffer/bandwidth management at routers – Dijkstra’s shortest path in link-state routing • Huffman coding algorithm • Prim’s algorithm for Minimum Spanning Tree • Selection on a column store 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 10

std: : priority_queue #include <queue> int main() Derek Carol Bob Alice { 20 11 9 4 priority_queue<string> q 1; q 1. push("Bob"); q 1. push("Carol"); q 1. push("Alice"); q 1. push("Derek"); while (!q 1. empty()) { cout << q 1. top() << " "; q 1. pop(); } cout << endl; priority_queue<int> q 2; q 2. push(9); q 2. push(20); q 2. push(11); q 2. push(4); while (!q 2. empty()) { cout << q 2. top() << " » ; q 2. pop(); } cout << endl; } But what about objects without < operator built in? Or want priority to be reversed? 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 11

Detour: std: : stack is a container adaptor • stack<int> stack 1; // deque is the container • stack<int, list<int>> stack 2; // list is the container • stack<int, vector<int>> stack 3; // vector is the container 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 12

std: priority_queue Container class Comparator functor std: : greater priority_queue<string, vector<string>, greater<string> > q 1; priority_queue<int, vector<int>, greater<int> > q 2; 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 13

Custom Objects struct Student { string name; unsigned int age; Student(string n="", unsigned int a=18) : name(n), age(a) {} }; name as priority age as priority 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 14

Compilation error if we do this int main() { // in name_pq, priorities are by name priority_queue<Student> pq 1; pq 1. push(Student("Bob", 18)); pq 1. push(Student("Carol", 17)); pq 1. push(Student("Alice", 16)); pq 1. push(Student("Derek", 15)); while (!q 1. empty()) { cout << "[" << pq 1. top(). name << ", " << pq 1. top(). age() << "] "; pq 1. pop(); } cout << endl; } invalid operands to binary expression ('const Student' and 'const Student') {return __x < __y; } 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 15

3 ways out • Overload < operator for Student class – Google style guide recommends against this • • Fool users into thinking expensive ops are cheap Hard to grep for them Some operators work on pointers too (like +) … • Functor • Function pointer 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 16

The functor way // this is a functor, or function object class Name. Comparator { bool operator()(const Student & x, const Student & y) { return (x. name < y. name); } }; // in name_pq, priorities are by name priority_queue<Student, vector<Student>, Name. Comparator> name_pq; name_pq. push(Student("Bob", 18)); name_pq. push(Student("Carol", 17)); name_pq. push(Student("Alice", 16)); name_pq. push(Student("Derek", 10)); while (!name_pq. empty()) { cout << "[" << name_pq. top(). name << ", " << name_pq. top(). age << "] "; name_pq. pop(); } 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 17

The function pointer way // this is the alternative way: straight function typedef bool (*Age. Comparator)(const Student &, const Student &); bool younger(const Student & x, const Student & y) { return (x. age < y. age); } // in age_pq, priorities are by age priority_queue<Student, vector<Student>, Age. Comparator> age_pq(younger); age_pq. push(Student("Bob", 18)); age_pq. push(Student("Carol", 17)); age_pq. push(Student("Alice", 16)); age_pq. push(Student("Derek", 10)); while (!age_pq. empty()) { cout << "[" << age_pq. top(). name << ", " << age_pq. top(). age << "] "; age_pq. pop(); } 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 18

How to implement a PQ with what’s known • Using an unsorted std: : vector – Push takes O(1) – Pop, Top take O(n) • Using a sorted std: : vector – Push takes O(n) – Pop, Top take O(1) • Similarly for a std: : list • O(n) is way too slow for high-speed routers! 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 19

Need New Designs! 10/31/2020 CSE 250, Fall 2012, SUNY Buffalo, © Hung Q. Ngo 20

Illustration of a binary tree in use - Heap sort algorithm - Priority Queues HEAP DATA STRUCTURE AND HEAP SORT ALGORITHM 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 21

Binary Max heap Complete binary tree - Every level is filled - Except possibly the last Node’s key ≥ children’s keys Search? 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 22

Max heap as an array 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 23

Heap sort • heapify: turn a vector/array into a heap • sink(i, n, array) – Makes sub-tree rooted at i a max heap – Assumes left & right sub-trees are already max heap – Sinks node i down to the correct level • heap_sort(array): – heapify(array) – swap root to array[n] – sink(0, n-1, array) 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 24

Heapify and heap_sort void heapify(vector<int> &vec) { for (int i=vec. size()/2; i>=0; i--) sink(vec, i, vec. size()); } void heap_sort(vector<int> &vec) { heapify(vec); for (int j=vec. size()-1; j>=1; j--) { swap(vec[0], vec[j]); sink(vec, 0, j); } } 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 25

Sinking, Recursively void recursive_sink(vector<int> &vec, size_t i, size_t n) { size_t left = 2*i + 1; if (n > vec. size() || left >= n) return; size_t right = left + 1; // possibly >= n size_t my_pick = (right >= n) ? left : (vec[right] > vec[left]) ? right : left; if (vec[i] < vec[my_pick]) { swap(vec[i], vec[my_pick]); recursive_sink(vec, my_pick, n); } } 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 26

Time and Space • sinking a node at height h takes c(h+1) time • heapify sinks nodes at various heights, O(n) time • heap_sort() – runs in time O(n log n) – space complexity O(log n) due to recursion 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 27

Sinking Iteratively void iterative_sink(vector<int> &vec, size_t i, size_t n) { if (n > vec. size()) return; size_t left, right, my_pick; while ((left = 2*i+1) < n) { right = left + 1; // possibly >= n my_pick = right >= n ? left : vec[right] > vec[left] ? right : left; if (vec[i] >= vec[my_pick]) break; swap(vec[i], vec[my_pick]); i = my_pick; } } 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 28

Binary Heap as Priority Queue • Insert new key takes O(log n) • Delete key takes O(log n) • Drawback: search takes a long time • Extremely simple to implement 10/31/2020 CSE 250, SUNY Buffalo, © Hung Q. Ngo 29