CS 315 Goals Heap Chapter 6 priority queue

CS 315 Goals: • Heap (Chapter 6) • priority queue • definition of a heap • Algorithms for • Insert • Delete. Min • percolate-down • Build-heap April 29, 13

Priority queue • primary operations: • Insert • delete. Min • secondary operations: • decrease. Key(j, d) • decrease the key in position (index) j by d (d > 0) • increase. Key(j, d) • increase thekey in position (index) j by d (d > 0) • Delete(j) • delete key stored in position (index) j Note: deleting a given key x is not one of the operations in our list (it takes O(n) time). • Performance goal: • O(log n) for all operations (worst-case) • Alternative to AVL tree. (far simpler)

Array implementation of priority queue As a first attempt, let us try: • option 1: keep the array sorted • insert O(n) time • delete. Min O(1) time • option 2: keep the array unsorted • insert O(1) time • delete. Min O(n) time Both options require O(n) operations for one of the two operations and so do not meet the requirement of O(log n) in the worst-case for both operations. Solution: keep the array semi-sorted, i. e. , some order relation is imposed that is flexible enough to support insert and delete. Min.

Binary Heap as an array and as a tree Array of 10 keys stored in A[1] through A[10] Can be viewed as a tree. Index j has index 2 j and 2 j+1 as children.

Connection between size and height of a heap h=1 h=2 h=3 In general, if the number of nodes is N, height is at most = O(log N)

Max-Heap property Key at each node must be bigger than or equal to its two children. Min-heap: parent key < = child key. Left-child can be < or = or > than right-child. See above example.

Min. Heap and non-Min Heap examples Violates Min. Heap property 21>6 13 13 21 24 65 31 26 19 26 6 65 31 26 19 32 19 68 Not a Heap 13 16 31 16 Not a Heap 21 65 68 A Min. Heap 32 13 24 21 16 21 68 32 Violates heap structural property 24 65 26 16 31 19 32 Violates heap structural property

Heap class definition class Binary. Heap {private: vector<Comparable> array; int current. Size; public: // member functions } A 13 27 4 7 58 3 4 1 2 3 4 5 3 7 4 7 8 Note: current. Size is different from the size of the vector (which is the number of allocated memory cells).

Heap Insertion Example 13 3 4 27 47 Insert 2 into the heap Create a hole and percolate it up. 58 Size = 6, so initially, j= 6; Since H[j/2]= 4 > 2, H[3] is copied to H[6]. Thus the heap now looks as follows: The new value of j=3. Since H[j/2] =H[1] = 3 is also greater 13 than 2, H[1] copied to H[3]. Now the heap looks as in the 3 27 next slide. 47 58 4 6

Heap Insertion Example continued 1 The new value of j = [3/2] = 1. 2 7 4 7 3 3 Now, j/2 = 0 so the iteration stops. 58 4 6 The last line sets H[j] = H[1] = 2. The final heap looks as follows: 12 2 7 4 7 3 3 5 8 4 6

Code for insert void insert(const Comparble & x) { if (current. Size == array. size() – 1) array. re. Size(array. size() * 2); //percolate up int hole = ++current. Size; for (; hole > 1 && x < array[ hole / 2] ; hole/= 2) array[ hole ] = array[ hole / 2 ]; array[ hole ] = x; } Try some more examples and make sure that the code works correctly.

Complexity of insert operation The height of a heap with n nodes is O(log n). The insert procedure percolates up along a path from a leaf to the root. Along the path at each level, one comparison and one data movement is performed. Conclusion: Insert performs O (log n) elementary operations (comparisons and data movements) to insert a key into a heap of size n in the worst-case.

Delete. Min Operation • Observation: The minimum key is at the root. • Find. Min() can be performed in O(1) time: Comparable find. Min() { if (current. Size == 0) cout << “heap is empty. ” << endl; else return array[1]; } • Deleting this key creates a hole at the root and we perform percolate down to fill the hole.

Example of Delete. Min 1 12 2 11 47 2 11 3 4 5 8 14 6 4 7 3 4 5 8 14 6 After deleting the min key, the hole is at the root. How should we fill the hole? We also need to vacate the index 6 since the size of the heap now reduces to 5. We need to find the right place for key 14.

Example of Delete. Min 2 11 7 3 4 5 8 14 6 Hole should be filled with smallest key, and it is in array[2] or array[3].

Example of Delete. Min 4 1 2 11 7 2 11 3 4 5 8 14 6 7 3 5 8 14 6 4 2 11 7 5 8 3 14 Try some more examples

Delete. Min – Outline of general case i Vacated key = k x L ? y R Hole is currently in index i. its children are 2 i and 2 i+1. Suppose array[2 i] = x, array[2 i+1]= y. • Case 1: (k <= x) && (k <= y) In this case, put k in array[I] and we are done. • Case 2: (k > x) && (y > x). Thus, x is the minimum of the three keys k, x and y. We move the key x up so the hole percolates to 2 i. Now the hole percolates down to 2 i. • Case 3: (k >x) && (x >y). Move y up and the hole percolates down to 2 i+1. • Case 4: the hole is at a leaf. put k there and this terminates

Code for delete-min /** * Remove the minimum item and place it in min. Item. * Throws Underflow if empty. */ void delete. Min( Comparable & min. Item ) { if( is. Empty( ) ) throw Underflow. Exception( ); min. Item = array[ 1 ]; array[ 1 ] = array[ current. Size-- ]; percolate. Down( 1 ); }

Code for percolate. Down void percolate. Down( int hole ) { int child; Comparable tmp = array[ hole ]; for( ; hole * 2 <= current. Size; hole = child ) { child = hole * 2; if( child != current. Size && array[ child + 1 ] < array[ child ] ) child++; if( array[ child ] < tmp ) array[ hole ] = array[ child ]; else break; } array[ hole ] = tmp; }

Code for percolate. Down void percolate. Down( int hole ) { int child; Comparable tmp = array[ hole ]; for( ; hole * 2 <= current. Size; hole = child ) { child = hole * 2; if( child != current. Size && array[ child + 1 ] < array[ child ] ) child++; if( array[ child ] < tmp ) array[ hole ] = array[ child ]; else break; } array[ hole ] = tmp; } Summary: percolate. Down(j) can be called when the heap property is violated at node j in that array[j] could be > array[2 j] or array[2 j+1]. (All the nodes in the subtree rooted at j satisfy the heap property. ) After the call, the heap property is satisfied at all the nodes in the subtree rooted at j.

increase. Key operation Suppose the priority of a key stored in position p changes. Specifically, we want to increase the key in A[p] by quantity d. Example: Change this to 22 13 21 24 65 26 16 31 32 p = 3, d = 6 New key is 22 19 68 After changing the key to 22, call percolate. Down(3)

increase. Key operation Example: percolate. Down(3) 13 13 21 24 65 26 22 31 32 19 21 68 24 65 26 19 31 32 22 68

increase. Key operation void increase. Key(int j, int d) { // change array[j] to array[j]+d and restore heap // d >= 0 is a pre-condition array[j] += d; percolate. Down(j); }

decrease. Key operation In this case, we need to percolate the changed key up the heap. Example: j = 4 and d = 12. Now, it is possible that array[j] is < array[j / 2] so we need to move the key up until the correct place is found. 12 13 Change to 12 21 24 65 26 16 31 32 13 19 21 68 65 26 16 31 32 19 68

decrease. Key operation void decrease. Key(int j, int d) { // change array[j] to array[j] – d and restore heap // d >= 0 is a pre-condition int temp = array[j]- d; while (j > 1 && array[j/2] > temp){ array[j] = array[j/2]; j/= 2; }; array[j] = temp; } Note that this procedure is very similar to insert (shown below): void insert(const Comparble & x) { if (current. Size == array. size() – 1) array. re. Size(array. size() * 2); //percolate up int hole = ++current. Size; for (; hole > 1 && x < array[ hole / 2]; hole/= 2) array[ hole ] = array[ hole / 2 ]; array[ hole ] = x; }

Removing a key at index p in a heap Example: Remove key at index 5 13 21 24 65 26 One way to do this: 22 31 19 68 32 Remove key What is the resulting heap? • apply decrease. Key to make the key the smallest. (Ex: d = 32) • perform delete. Min

Converting an array into a heap We pass a vector items containing n (arbitrary) keys to the heap class constructor whose task it is to store these keys as a heap. • Read these n keys into the heap array. • call a procedure build. Heap to convert array into a heap. binary. Heap(const vector<int> & items) { int n = items. size(); for (int i = 0; i < n; ++i) array[i+1] = items[i]; build. Heap(n); }

Converting an array into a heap Starting with an arbitrary array, how to convert it to a heap? Solution 1: We can view the array as a heap of size 1. Successively insert keys array[2], array[3], etc. into the heap until all the keys are inserted. void build. Heap(int n) { current. Size = 1; for (int j= 2; j <= n; ++j) { insert(array[j]); } }; This takes O(n log n) time. (n operations each taking O(log n) time. )

Alternate (faster) way to build. Heap Idea: perform percolate. Down(j) starting from j to j = 1. = current. Size/2 Why does it work? • Before percolate. Down(j) is called, we have already performed percolate. Down at 2 j and 2 j+1, and so in the subtree rooted at j, the only node where the heap property may not hold is j. • Thus, after percolate. Down(j) is called, the subtree rooted at j is a heap. • The last call is percolate. Down(1) so the tree rooted at 1 is a heap at the end. Code for solution 2: void build. Heap 2() { n = array. size(); current. Size = n; for (int j = n/2; j > 0; --j) percolate. Down(j); }

Build. Heap using percolate. Down is faster Solution 2 builds a heap in O(n) time. Analysis of this faster algorithm is presented in the text. We will skip it since it is complicated. (Involves summation of a series called the arithmeticgeometric series. )

Heap-sorting To sort n numbers: • read the numbers into an array A. • call heap constructor to build a heap from A. • perform n delete. Min operations. Store the successive outputs of delete. Min in the array A. • A is now sorted. Total number of operations = O(n) (for builing heap) + O(n log n) (for n delete. Min operations) = O(n log n)

Heap-sorting void sort(vector<Comparable>& out) { // the sorted array is output in array out. resize(current. Size); int j=0; while (!is. Empty()) out[j++] = delete. Min(); } It is possible to collect the output in the heap array itself. As the heap size is shrinking, we can use the unused slots of the array to collect the delete min keys. The resulting heap will be sorted in descending order.

Summary of heap • binary heap is implemented as an array. (Tree representation is conceptual – for understanding, not explicit. ) • heap data members are: array (a vector) and current. Size. • efficient heap operations: insert, findmin, deletemin, decrease. Key, increase. Key – all of which can be performed in O(log n), findmin can be done in O(1) in the worst-case. • an important helper function is percolate. Down. It also takes O(log n) time.

Summary of heap • build-heap is the process of converting a set of keys into a heap-ordered array. • build-heap can be performed in O(n log n) time by a series of n – 1 insertions. • faster build-heap can be performed in O(n) time using n/2 percolate-down operations. • heap-sorting takes O(n log n) in the worst-case • Step 1: build a heap • Step 2: perform n – 1 delete. Min operations and put the key at the end of the heap-array.

Summary of heap • Some heap operations are expensive (i. e. , can take n steps in the worst-case): • search(x) – search for a key x. • delete(x) – delete the key x (if present) in a heap • Priority queue has many applications • Example: scheduling problem, shortest-path problem (Dijkstra’s algorithm) etc.