Priority Queues and Heaps CMPT 225 Remember Queues

Priority Queues and Heaps CMPT 225

Remember Queues? A queue should implement at least the first two of these operations: insert – insert item at the back of the queue remove – remove an item from the front peek – return the item at the front of the queue without removing it It is assumed that these operations will be implemented efficiently That is, in constant time October 2004 John Edgar 2

Queue Implementations �Either with an array 4 0 18 25 1 2 2 5 3 4 5 6 7 6 �Or with a linked list front back 4 7 3 October 2004 John Edgar 3

FIFO �Queues are first-in first-out (FIFO) �Priority queues are a fancier type of queue Maintains an ordering of items in the queue, not necessarily first-in first-out October 2004 John Edgar 4

Priority Queues 4 5 1 6 September 2004 John Edgar 2 3 5

Priority Queues �Items in a priority queue are given a priority value Which could be numerical or something else �The highest priority item is removed first � Uses include System requests Data structure to support Dijkstra’s Algorithm September 2004 John Edgar 6

Priority Queue Problem �Can items be inserted and removed efficiently from a priority queue? Using an array, or Using a linked list? �Note that items are not removed based on the order in which they are inserted Now we’ll see how we can do these efficiently (using a different data structure) September 2004 John Edgar 7

ADT Priority Queue �Items in a priority queue have a priority Not necessarily numerical Could be lowest first or highest first �The highest priority item is removed first �Priority queue operations Insert Remove in priority queue order ▪ Both operations should be performed in at most O(log n) time October 2004 John Edgar 8

Priority Queue Implementation September 2004 John Edgar 9

Implementing a Priority Queue � Items have to be removed in priority order This can only be done efficiently if the items are ordered in some way � One option would be to use a balanced binary search tree Binary search trees are fully ordered and insertion and removal can be implemented in O(log n) time ▪ Some operations (e. g. removal) are complex ▪ Although operations are O(logn) they require quite a lot of structural overhead � There is a much simpler binary tree solution October 2004 John Edgar 10

Heaps � A heap is binary tree with two properties � Heaps are complete All levels, except the bottom, are completely filled in The leaves on the bottom level are as far to the left as possible. � Heaps are partially ordered The value of a node is at least as large as its children’s values, for a max heap or The value of a node is no greater than its children’s values, for a min heap October 2004 John Edgar 11

Complete Binary Trees complete binary trees incomplete binary trees October 2004 John Edgar 12

Partially Ordered Tree – max heap 98 86 41 13 9 65 10 44 32 23 21 29 17 Heaps are not fully ordered, an inorder traversal would result in: 9, 13, 10, 86, 44, 65, 23, 98, 21, 32, 17, 41, 29 October 2004 John Edgar 13

Priority Queues and Heaps � A heap can implement a priority queue � The item at the top of the heap must always be the highest priority value Because of the partial ordering property � Implement priority queue operations: Insertions – insert an item into a heap Removal – remove and return the heap’s root For both operations preserve the heap property October 2004 John Edgar 14

Heap Implementation Using an Array September 2004 John Edgar 15

Heap Implementation � Heaps can be implemented using arrays � There is a natural method of indexing tree nodes 0 Index nodes from top to bottom and left to right as shown on the right Because heaps are complete binary trees there can be no gaps in the array October 2004 John Edgar 2 1 3 4 5 6 16

Referencing Nodes �It will be necessary to find the index of the parents of a node Or the children of a node �The array is indexed from 0 to n – 1 Each level's nodes are indexed from: ▪ 2 level – 1 to 2 level+1 – 2 (where the root is level 0) The children of a node i, are the array elements indexed at 2 i + 1 and 2 i + 2 The parent of a node i, is the array element indexed at floor((i – 1) / 2) October 2004 John Edgar 17

Heap Array Example 98 86 Heap 13 9 7 41 65 10 1 1 3 8 0 44 9 4 32 23 10 21 2 5 11 29 17 6 12 index 0 2 3 4 5 6 7 8 9 10 11 12 value 98 86 41 13 65 32 29 9 10 44 23 21 17 Underlying Array October 2004 John Edgar 18

Heap Insertion September 2004 John Edgar 19

Heap Insertion � On insertion the heap properties have to be maintained; remember that A heap is a complete binary tree and A partially ordered binary tree � There are two general strategies that could be used to maintain the heap properties Make sure that the tree is complete and then fix the ordering or Make sure the ordering is correct first Which is better? October 2004 John Edgar 20

Heap Insertion Sketch � The insertion algorithm first ensures that the tree is complete Make the new item the first available (left-most) leaf on the bottom level i. e. the first free element in the underlying array � Fix the partial ordering Compare the new value to its parent Swap them if the new value is greater than the parent Repeat until this is not the case ▪ Referred to as bubbling up, or trickling up October 2004 John Edgar 21

Heap Insertion Example Insert 81 98 86 41 13 9 65 10 2 44 3 23 5 17 0 6 7 8 10 11 12 value 98 86 41 13 65 32 29 9 10 44 23 21 17 John Edgar 4 21 29 index October 2004 1 32 9 13 22

Heap Insertion Example Insert 81 81 is less than 98 so finished 98 81 41 86 13 9 65 10 32 44 23 21 29 81 41 17 29 81 (13 -1)/2 = 6 index 0 6 7 8 10 11 12 13 value 98 86 41 81 13 65 32 29 41 81 9 10 44 23 21 17 29 81 October 2004 1 2 3 John Edgar 4 5 9 23

Heap Removal September 2004 John Edgar 24

Heap Removal � Make a temporary copy of the root’s data � Similarly to the insertion algorithm, first ensure that the heap remains complete Replace the root node with the right-most leaf i. e. the highest (occupied) index in the array � Swap the new root with its largest valued child until the partially ordered property holds i. e. bubble down � Return the root’s data October 2004 John Edgar 25

Heap Removal Example Remove (root) 98 86 41 13 9 65 10 23 21 29 17 index 0 2 3 4 5 6 7 8 9 10 11 12 value 98 86 41 13 65 32 29 9 10 44 23 21 17 October 2004 1 44 32 John Edgar 26

Heap Removal Example Remove (root) replace root with right-most leaf 98 17 86 41 13 9 65 10 23 21 29 17 index 0 2 3 4 5 6 7 8 9 10 11 12 value 98 17 86 41 13 65 32 29 9 10 44 23 21 17 October 2004 1 44 32 John Edgar 27

Heap Removal Example ? Remove (root) ? 86 17 17 86 41 13 9 65 10 1 swap with larger child 44 32 23 29 21 index 0 2 3 4 5 6 7 8 9 10 11 value 86 17 41 13 65 32 29 9 10 44 23 21 12 children of root: 2*0+1, 2*0+2 = 1, 2 October 2004 John Edgar 28

Heap Removal Example Remove (root) ? ? 17 65 13 9 swap with larger child 86 41 65 17 10 44 32 23 29 21 index 0 1 2 3 4 5 6 7 8 9 10 11 value 86 65 17 41 13 65 17 32 29 9 10 44 23 21 12 children of 1: 2*1+1, 2*1+2 = 3, 4 October 2004 John Edgar 29

Heap Removal Example Remove (root) swap with larger child 86 65 13 9 41 44 17 ? 10 44 17 32 ? 23 29 21 index 0 1 2 3 4 5 6 7 8 9 10 11 value 86 65 41 13 44 17 32 29 9 10 44 17 23 21 12 children of 4: 2*4+1, 2*4+2 = 9, 10 October 2004 John Edgar 30

Bubbles September 2004 John Edgar 31

Bubble Up and Bubble Down � Helper functions are usually written for preserving the heap property bubble. Up ensures that the heap property is preserved from the start node up to the root bubble. Down ensures that the heap property is preserved from the start node down to the leaves � These functions may be implemented recursively or iteratively October 2004 John Edgar 32

Bubble. Down Algorithm Go to terminal October 2004 John Edgar 33

Removal Algorithm October 2004 John Edgar 34

Insertion Algorithm �Lab next week October 2004 John Edgar 35

Bubble. Up Algorithm �Lab next week October 2004 John Edgar 36

Heap Efficiency � For both insertion and removal the heap performs at most height swaps For insertion at most height comparisons ▪ To bubble up the array For removal at most height * 2 comparisons ▪ To bubble down the array (have to compare two children) � Height of a complete binary tree is log 2(n) Both insertion and removal are therefore O(logn) October 2004 John Edgar 37

Sorting with Heaps September 2004 John Edgar 38

Sorting with Heaps � Observation 1: Removal of a node from a heap can be performed in O(logn) time � Observation 2: Nodes are removed in order � Conclusion: Removing all of the nodes one by one would result in sorted output � Analysis: Removal of all the nodes from a heap is a O(n*logn) operation October 2004 John Edgar 39

But … � A heap can be used to return sorted data In O(n*logn) time � However, we can’t assume that the data to be sorted just happens to be in a heap! Aha! But we can put it in a heap. Inserting an item into a heap is a O(logn) operation so inserting n items is O(n*logn) � But we can do better than just repeatedly calling the insertion algorithm October 2004 John Edgar 40

Heapifying Data � To create a heap from an unordered array repeatedly call bubble. Down Any subtree in a heap is itself a heap Call bubble. Down on elements in the upper ½ of the array Start with index n/2 and work up to index 0 ▪ i. e. from the last non-leaf node to the root � bubble. Down does not need to be called on the lower half of the array (the leaves) Since bubble. Down restores the partial ordering from any given node down to the leaves October 2004 John Edgar 41

Heapify Example Assume unsorted input is contained in an array as shown here (indexed from top to bottom and left to right) 89 0 29 1 36 27 October 2004 7 John Edgar 3 70 23 48 8 76 9 4 94 37 10 42 11 5 58 2 13 6 12 42

Heapify Example n = 12, (n-1)/2 = 5 bubble. Down(5) 89 0 29 1 36 27 October 2004 7 John Edgar 3 70 23 48 8 76 9 4 94 37 10 42 11 5 58 2 13 6 12 43

Heapify Example n = 12, (n-1)/2 = 5 bubble. Down(5) 89 bubble. Down(4) 0 29 1 36 27 October 2004 7 John Edgar 3 70 23 48 76 8 48 76 9 4 94 37 10 42 11 5 58 2 13 6 12 44

Heapify Example n = 12, (n-1)/2 = 5 bubble. Down(5) 89 bubble. Down(4) 29 1 bubble. Down(3) 70 36 27 October 2004 0 7 John Edgar 3 36 70 23 76 8 48 9 4 94 37 10 42 11 5 58 2 13 6 12 45

Heapify Example n = 12, (n-1)/2 = 5 bubble. Down(5) 89 bubble. Down(4) 94 23 29 1 bubble. Down(3) bubble. Down(2) 70 27 October 2004 0 7 John Edgar 3 36 76 8 48 9 4 23 94 58 37 10 42 11 5 58 23 2 13 6 12 46

Heapify Example n = 12, (n-1)/2 = 5 bubble. Down(5) 89 bubble. Down(4) 29 76 1 bubble. Down(3) bubble. Down(2) 70 bubble. Down(1) 27 October 2004 0 7 John Edgar 3 36 94 76 29 48 8 48 29 9 4 58 37 10 42 11 5 23 2 13 6 12 47

Heapify Example n = 12, (n-1)/2 = 5 bubble. Down(5) 94 89 bubble. Down(4) 0 89 94 76 1 bubble. Down(3) bubble. Down(2) 70 bubble. Down(1) 3 48 4 58 5 2 13 6 bubble. Down(0) 27 October 2004 7 John Edgar 36 8 29 9 37 10 42 11 23 12 48

Cost to Heapify an Array � bubble. Down is called on half the array The cost for bubble. Down is O(height) It would appear that heapify cost is O(n*logn) � In fact the cost is O(n) � The analysis is complex but bubble. Down is only called on n/2 nodes and mostly on sub-trees October 2004 John Edgar 49

Heap. Sort Algorithm Sketch � Heapify the array � Repeatedly remove the root After each removal swap the root with the last element in the tree The array is divided into a heap part and a sorted part � At the end of the sort the array will be sorted in reverse order October 2004 John Edgar 50

Heap. Sort Notes �The algorithm runs in O(n*logn) time Considerably more efficient than selection sort and insertion sort The same average case complexity as Merge. Sort and Quick. Sort �The sort can be carried out in-place That is, it does not require that a copy of the array to be made October 2004 John Edgar 51

Summary September 2004 John Edgar 52

Objectives �Define the ADT priority queue �Define the partially ordered property �Define a heap �Implement a heap using an array �Implement the heap. Sort algorithm October 2004 John Edgar 53

Readings �Java Ch. 12 �C++ Ch. 11 October 2004 John Edgar 54