HEAPS Amihood Amir Bar Ilan University 2014 Sorting

HEAPS Amihood Amir Bar Ilan University 2014

Sorting Bubblesort: Until no exchanges: For i=1 to n-1 if A[i]>A[i+1] then exchange their values end Time: O(n 2)

"Bubbling Up" the Largest Element Traverse a collection of elements – Move from the front to the end – “Bubble” the largest value to the end using pair-wise comparisons and swapping 1 2 3 4 5 6 42 Swap 77 42 77 35 12 101 5

"Bubbling Up" the Largest Element Traverse a collection of elements – Move from the front to the end – “Bubble” the largest value to the end using pair-wise comparisons and swapping 1 42 2 3 35 Swap 35 77 77 4 5 6 12 101 5

"Bubbling Up" the Largest Element Traverse a collection of elements – Move from the front to the end – “Bubble” the largest value to the end using pair-wise comparisons and swapping 1 2 42 35 3 4 12 Swap 12 77 77 5 6 101 5

"Bubbling Up" the Largest Element Traverse a collection of elements – Move from the front to the end – “Bubble” the largest value to the end using pair-wise comparisons and swapping 1 2 3 4 5 6 42 35 12 77 101 5 No need to swap

"Bubbling Up" the Largest Element Traverse a collection of elements – Move from the front to the end – “Bubble” the largest value to the end using pair-wise comparisons and swapping 1 2 3 4 42 35 12 77 5 6 5 Swap 101 5

"Bubbling Up" the Largest Element Traverse a collection of elements – Move from the front to the end – “Bubble” the largest value to the end using pair-wise comparisons and swapping 1 2 3 4 5 6 42 35 12 77 5 101 Largest value correctly placed

Sorting Selection sort: For i=1 to n do For j=i+1 to n do If A[i]>A[j] then exchange them end Time: = O(n 2)

Selecting the Smallest Element Traverse a collection of elements – Move from the front to the end – Select the smallest value using pair-wise comparisons and swapping 1 2 3 4 5 6 42 Swap 77 42 77 35 12 101 5

Selecting the Smallest Element Traverse a collection of elements – Move from the front to the end – Select the smallest value using pair-wise comparisons and swapping 1 Swap 2 3 4 5 6 35 42 77 42 35 12 101 5

Selecting the Smallest Element Traverse a collection of elements – Move from the front to the end – Select the smallest value using pair-wise comparisons and swapping 1 35 12 Swap 2 3 77 42 4 5 6 35 12 101 5

Selecting the Smallest Element Traverse a collection of elements – Move from the front to the end – Select the smallest value using pair-wise comparisons and swapping 1 12 2 77 3 4 42 35 No need to swap 5 6 101 5

Selecting the Smallest Element Traverse a collection of elements – Move from the front to the end – Select the smallest value using pair-wise comparisons and swapping 1 12 5 2 77 3 42 Swap 4 35 5 6 101 12 5

Selecting the Smallest Element Traverse a collection of elements – Move from the front to the end – Select the smallest value using pair-wise comparisons and swapping 1 2 5 77 3 4 42 35 Smallest value correctly placed 5 6 101 12

Sorting Insertion sort: For i=1 to n do Insert A[i+1] into appropriate (sorted) position in A[1], …, A[i] end Time: = O(n 2)

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 77 42 35 12 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 42 77 35 12 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 42 77 35 12 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 42 77 35 12 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 42 35 77 12 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 35 42 77 12 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 35 42 77 12 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 35 42 12 77 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 35 12 42 77 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 35 42 77 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 35 42 77 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 35 42 77 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 35 42 77 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 35 42 77 101 5

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 35 42 77 5 101

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 35 42 5 77 101

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 35 5 42 77 101

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 12 5 35 42 77 101

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 5 12 35 42 77 101

Keeping Prefix Sorted Traverse a collection of elements – Move from the front to the end – Keep the prefix sorted throughout 1 2 3 4 5 6 5 12 35 42 77 101

Complexity The Time of all algorithms we saw: O(n 2) Can we do better?

Merge Sort Based on the Merging operation. Given two sorted arrays: A[1], …, A[n] and B[1], …, B[m]. merge them into one sorted array: C[1], …, C[n+m]

Merging Pa, Pb, Pc <- 1 While (Pa < n+1 and Pb < m+1) If A[Pa] ≤ B[Pb] then C[Pc] <- A[Pa] Pa <- Pa+1 Pc <- Pc+1 else C[Pc] <- B[Pb] Pb <- Pb+1 Pc <- Pc+1 end

Merging If Pa=n+1 and Pb < m+1 then for i=Pb to m do C[Pc] <- B[i] Pc <- Pc +1 end If Pb=n+1 and Pa < m+1 then for i=Pa to n do C[Pc] <- A[i] Pc <- Pc +1 end Algorithm

Merging A: C: 3 10 23 54 B: 1 5 25 75

Merging A: C: 10 23 54 B: 3 1 5 25 75

Merging A: C: 10 23 54 B: 1 3 5 25 75

Merging A: C: 10 23 54 B: 1 3 5 25 75

Merging A: C: 23 54 B: 1 3 5 10 25 75

Merging A: C: 54 B: 1 3 5 10 23 25 75

Merging A: C: 54 B: 1 3 5 10 23 25 75

Merging A: C: B: 1 3 5 10 23 25 54 75

Merging A: C: B: 1 3 5 Time: O(n+m) 10 23 25 54 75 Linear !!!

Merge Sort A recursive sorting algorithm: Mergesort(A) If n=1 then Return(A) else Split A[1], …, A[n] to two length n/2 arrays: A[1], …, A[n/2] and A[n/2+1], …, A[n] Mergesort(A[1], …, A[n/2]) Mergesort(A[n/2+1], …, A[n]) Merge(A[1], …, A[n/2], A[n/2+1], …, A[n], B) A <- B Return(A)

Merge Sort Example 99 6 86 15 58 35 86 4 0

Merge Sort Example 99 99 6 6 86 15 58 35 86 4 4 0 0

Merge Sort Example 99 99 99 6 6 6 86 15 58 35 86 86 15 58 35 4 0 86 4 0

Merge Sort Example 99 99 6 6 86 15 58 35 86 86 15 4 0 58 35 86 4 58 86 35 0 4 0

Merge Sort Example 99 99 6 6 86 15 58 35 86 86 15 4 0 58 35 86 4 58 86 35 0 4 4 0 0

Merge Sort Example 99 6 Merge 86 15 58 35 86 0 4 4 0

Merge Sort Example 6 99 99 6 Merge 15 86 86 15 58 35 0 58 86 35 4 86 0 4

Merge Sort Example 6 6 99 Merge 15 86 99 15 86 0 4 58 35 35 58 86 0 4 86

Merge Sort Example 0 6 Merge 4 6 15 35 58 86 86 99 15 86 99 0 4 35 58 86

Merge Sort Example 0 4 6 15 35 58 86 86 99

Merge Sort Time The recurrence: TM(n)=2 TM(n/2)+n TM(1)=1 Closed Form of TM(n): O(n log n)

A Data Structures Algorithm Sort array A using AVL Trees: For i=1 to n do: Insert A[i] into AVL tree For i=1 to n do: Extract smallest element from AVL tree and put in A[i]

Sorting Algorithm using AVL Can we extract the smallest element from AVL tree? -- Yes. Go all the way to the left. Time: O(log n) Therefore: Sorting Algorithm Time: O(n log n)

Priority Queue What property of the AVL Tree did we use? Only that we can find the minimum fast. A Priority Queue is a data structure that supports the following operations: INSERT (Q, k) EXTRACTMIN(Q, m)

Priority Queue Implementation Heap: A binary tree where every vertex has a key. For every vertex v, key(v) ≤ key(w), where w is a child of v. Difference between heap and binary search tree: 10 40 10 30 Heap 5 30 Binary search tree

Heap Additional Heap Property: All levels of the heap are full, except possibly the last level, which is left-filled. Example:

Heap Properties The keys on every path from the root to a leaf are nondecreasing. Example:

Heap Properties The height of a heap with n elements is O(log n). Example: h < log n ≤ h+1

Heap Insertion Insert 6 3 4 7 10 21 22 28 23 20 35 25 8 6

Heap Insertion Insert 6 3 4 7 10 21 22 28 23 20 6 35 25 8 20 6

Heap Insertion Insert 6 3 OK 4 7 6 10 21 22 28 23 8 6 7 35 25 20

Heap Deletion Delete Min 3 Now fix heap 20 3 4 6 10 21 22 28 23 8 7 35 25 20

Heap Deletion Delete Min 3 20 4 10 21 22 6 28 23 7 35 25 8

Heap Deletion Delete Min 3 4 20 10 6 10 20 21 7 OK 22 28 23 35 25 8

Heap Operations Time O(log n) Conclude: Heap of n elements can be constructed in time: O(n log n) But: We can do a lot better!!!

Build Heap Put all elements in a tree where all levels are full and the last level is left-filled. Construct heaps from leaves up

Time Analysis For trees in level h : For trees in level h-1 : 1 2 For trees in level h-i : i For trees in level 0 : log n

Time Analysis Number of trees in level h: n/2 Number of trees in level h-1: n/22 … … Number of trees in level h-i: n/2 i+1 … Number of trees in level 0 : 1

Total Time Calculate it: Note that n/2 trees do constant time work, and another quarter need to go to two levels. Only one tree needs to fix log n levels. Formally: Need to calculate this

Total Time We know: for x<1. Differentiate both sides: Multiply both sides by x:

Total Time In our case: x = 1/2. So total time: O(n) .

Look Ma No Pointers In our case: So write Our tree down is almost all keys complete. Allfrom sequentially levels are to top fullbottom and theand last one is left toleft-filled. right.

Getting rid of Pointers Example: 3 4 6 10 21 22 28 23 7 35 25 8 20 3, 4, 6, 21, 10, 7, 8, 22, 28, 23, 35, 20

Calculate Location node i in level j = location 2 j-1+i in array. 3 4 6 10 21 22 28 23 7 35 25 Location i in array = node level. 8 20 in

Location of children Node j in row i is the 2 i-1+j’s element. Its left son is the 2 i+1 -1+2(j-1)+1’s element. row i j But 2 i+1 -1+2(j-1)+1 = 2 i+1+2(j-1) = 2 i+1+2 j-2 = 2(2 i-1+j).

Location of Children Conclude: Location of left son of i is 2 i. Location of right son of i is 2 i+1. Location of father of i is: i/2, if i is even (i-1)/2, if i is odd

Heap Sort How? Given array A of n elements: Construct a heap HA in-place. For i=n downto 1 do EXTRACTMIN(HA, v) Put v in location A[i] Endfor Time: O(n) for heap construction and O(nlog n) for loop.

Heap Sort Note: Array A is now sorted in non-increasing order. What do we do if we want to sort in nondecreasing order? Options: 1. Reverse order of A in linear time inplace. 2. Use MAX rather than MIN priority Queue.