Sorting Chapter 7 Sorting Why do we want

Sorting Chapter 7

Sorting Why do we want data sorted? What advantages are there? 1

Sorting Why do we want data sorted? What advantages are there? Faster find operations using binary search Takes find from O(n) to O( log n) 2

Sorting takes effort Is this effort worth it? 3

Sorting takes effort Is this effort worth it? Definitely not for a single find But it is if you do enough finds 4

Sorting What sorts are you familiar with? 5

Sorting What sorts are you familiar with? Bubble Selection 6

Bubble Sort How does bubble sort work? 13 34 1 8 36 7 2 22

Bubble Sort How does bubble sort work? Loop through the array swapping adjacent elements if they are out of order After a loop, it looks like: 13 1 8 34 2 Are we done? 8 22 36

Bubble Sort How does bubble sort work? Loop through the array swapping adjacent elements if they are out of order After a loop, it looks like: 13 1 8 34 2 22 36 Are we done? No, how many passes do we have to make before we are done? 9

Bubble Sort How does bubble sort work? Loop through the array swapping adjacent elements if they are out of order After a loop, it looks like: 13 1 8 34 2 22 36 Are we done? No, how many passes do we have to make before we are done? N passes are required 10

Bubble Sort Do the second pass 13 1 8 34 2 11 22 36

Bubble Sort After the second pass, now do the third pass 1 8 13 2 22 12 34 36

Bubble Sort After the third pass, now do the fourth pass 1 8 2 13 22 13 34 36

Bubble Sort After the fourth pass, now do the fifth pass 1 2 8 13 22 14 34 36

Bubble Sort What is the complexity? 15

Bubble Sort What is the complexity? One pass of the array is n The first pass is n, the second n-1, the third n-2, … There are n passes of the array This is O(n*n) 16

Sort Analysis Sorts are not analyzed just on complexity Other criteria include: Internal vs. External Stable Oblivious 17

Sort Analysis Internal vs. External Internal does the sort in main memory This is most sorts External uses disk storage Less common, used when the data cannot fit in memory 18

Sort Analysis Stable Sorts A stable sort maintains order of elements with the same key For example: Smith, John Brown, Nick Young, Bob Smith, Kim Brown, Joe Smith, John Smith, Kim Young, Bob Sorted by last name: Brown, Nick Brown, Joe 19

Sort Analysis Why might stability be important? 20

Sort Analysis Why might stability be important? Because it allows you to sort by multiple keys. 21

Sort Analysis Sort by first name: Brown, Nick Brown, Joe Smith, John 22 Smith, Kim Young, Bob

Sort Analysis Sorted by first name, now sort by last name: Young, Bob Brown, Joe Smith, John 23 Smith, Kim Brown, Nick

Sort Analysis Sorted by first name, then by last: Notice the primary ordering is by last name, but within a last name, it stays ordered by first name Brown, Joe Brown, Nick Smith, John 24 Smith, Kim Young, Bob

Sort Analysis Oblivious Sorts An oblivious sort does the same amount of work for any array of the same size. An oblivious sort takes the same amount of time to sort both these arrays, even though one is already sorted and requires no work Smith, John Brown, Nick Young, Bob Smith, Kim Brown, Joe Brown, Nick Smith, John Smith, Kim Young, Bob 25

Sort Analysis Oblivious Sorts We want sorts to be Not Oblivious We want to take advantage of order that already exists Smith, John Brown, Nick Young, Bob Smith, Kim Brown, Joe Brown, Nick Smith, John Smith, Kim Young, Bob 26

Bubble Sort Internal vs. External? Stable? Oblivious? 27

Bubble Sort Internal vs. External? Internal Stable? Yes, you are only swapping adjacent elements, and you only swap if they are out of order Oblivious? Yes, at least in the basic form 28

Bubble Sort Internal vs. External? Internal Stable? Yes Oblivious? Yes Improvements? 29

Bubble Sort Improvements Make it Not Oblivious Keep a bool for if a swap occurred in the last pass If there were swaps, keep going If there were no swaps, that means nothing was out of order, thus it is in order, and so it can quit 30

Bubble Sort Now do it with the flag/bool for did. Swap = false 13 34 1 8 36 31 2 22

Bubble Sort Now do it with the flag/bool for did. Swap After one pass did. Swap =true 13 1 8 34 2 32 22 36

Bubble Sort Now do it with the flag/bool for did. Swap After two passes did. Swap =true 1 8 13 2 22 33 34 36

Bubble Sort Now do it with the flag/bool for did. Swap After three passes did. Swap =true 1 8 2 13 22 34 34 36

Bubble Sort Now do it with the flag/bool for did. Swap After four passes did. Swap =true 1 2 8 13 22 35 34 36

Bubble Sort Now do it with the flag/bool for did. Swap After five passes did. Swap =false 1 2 8 13 22 36 34 36

Bubble Sort Now, what is the complexity? 37

Bubble Sort Now, what is the complexity? Best case, the array is sorted, and there is one pass O(n) Worst case, the array is reversed and there are n passes O(n*n) Complexity is O(n*n) 38

Bubble Sort Code void bubble( Datatype* a, int size) { for(int i=0; i<size-1; i++) { bool did. Swap= false; for (int j = 0; j< size-1)-i; j++) if (a[j]. value > a[j+1]. value) { did. Swap = true; swap(a[j], a[j+1]); } if (!did. Swap) return; } } 39

Selection Sort How does Selection sort work? 13 34 1 8 36 40 2 22

Selection Sort How does Selection sort work? Select one element at a time and put it in the proper position Typically start with the smallest and work up Can start with the largest and work down 13 34 1 8 36 41 2 22

Selection Sort How does Selection sort work? After one element is put in final position 1 34 13 8 36 42 2 22

Selection Sort How does Selection sort work? After two elements are put in final position 1 2 13 8 36 43 34 22

Selection Sort How does Selection sort work? After three elements are put in final position 1 2 8 13 36 44 34 22

Selection Sort How does Selection sort work? After four elements are put in final position 1 2 8 13 36 45 34 22

Selection Sort How does Selection sort work? After five elements are put in final position 1 2 8 13 22 46 34 36

Selection Sort How does Selection sort work? After six elements are put in final position 1 2 8 13 22 47 34 36

Selection Sort How does Selection sort work? After seven elements are put in final position 1 2 8 13 22 48 34 36

Selection Sort Complexity? 49

Selection Sort Complexity? One search for the smallest is n The first search is n, the second n-1, the third n-2, … There are n searches for the smallest element This is O(n*n) 50

Selection Sort Code void Selection. Sort(Datatype* a, int size) { for (int pos= 0; pos< size; pos++) { int min. Index=pos; for(int i=pos+1; i<size; i++) if( a[i] < a[min. Index]) min. Index=i; swap(a[pos], a[min. Index]); } } 51

Selection Sort Internal vs. External? Stable? Oblivious? 52

Selection Sort Internal vs. External? Internal Stable? No, when you swap, you can swap things past other occurrences Oblivious? Yes, even if this element is in order, the rest of the array may not be 53

Selection Sort Stable? No, when you swap, you can swap things past other occurrences For example: Smith, John Brown, Nick Young, Bob 54 Smith, Kim Brown, Joe

Selection Sort Stable? No, when you swap, you can swap things past other occurrences After one element is sorted Brown, Nick Smith, John Young, Bob 55 Smith, Kim Brown, Joe

Selection Sort Stable? No, when you swap, you can swap things past other occurrences After two elements are sorted John Smith got swapped past Kim Smith Brown, Nick Brown, Joe Young, Bob 56 Smith, Kim Smith, John

Selection Sort Oblivious? Yes, even if this element is in order, the rest of the array may not be Remember our array from before This is after three elements are in their final position 1 2 8 13 36 34 22 When we go to do the fourth, no swaps are needed, but we don’t know if we can quit, because we don’t know if the rest of the array is in order 57

Bogo Sort Not a sort that is in your book You may hear about it though Algorithm Swap two random elements and check if it is sorted Repeat until it is sorted 58

Bogo Sort Complexity? Internal vs. External? Stable? Oblivious? 59

Insertion Sorts by inserting into a sorted array Done in place, meaning the sorted array is part of the array that is being sorted The sorted array is located at the beginning of the array 60

Insertion Sorts by inserting into a sorted array Start by saying the first element is sorted An array of size one is sorted This is the sorted array, everything after is is unsorted 13 34 1 8 36 61 2 22

Insertion Sorts by inserting into a sorted array Now, look at the second element Insert this into your sorted array (size 1) Save the value Push back any values that are greater Put the value in position In this case, no values are greater, so we are done Increase the size of your sorted array to 2 13 34 1 8 36 62 2 22

Insertion Sorts by inserting into a sorted array Now, look at the third element Insert this into your sorted array (size 2) Save the value Push back any values that are greater (both 34 and 13) Put the value in position (at element 0) Increase the size of your sorted array to 3 13 34 1 8 36 63 2 22

Insertion Sorts by inserting into a sorted array Now, look at the fourth element 1 13 34 8 36 64 2 22

Insertion Sorts by inserting into a sorted array Now, look at the fifth element 1 8 13 34 65 36 2 22

Insertion Sorts by inserting into a sorted array Now, look at the sixth element 1 8 13 34 66 36 2 22

Insertion Sorts by inserting into a sorted array Now, look at the seventh element 1 2 8 13 67 34 36 22

Insertion Sorts by inserting into a sorted array The array is sorted! 1 2 8 13 68 22 34 36

Insertion Sort Complexity? 69

Insertion Sort Complexity? On average, ½ of the sorted array gets pushed backward on an insert Each time an insert occurs, the size of the sorted array increases Work is: ½(1+2+3+4+…. +n) = n/2 * (n+1) ½(1+2+3+4+…. +n) = ½ * n/2 * (n+1) = ¼ * n(n+1) = ¼ n*n O(n*n) 70

Insertion Sort Code void Insertion. Sort(Datatype* a, int n) { for (int i = 1; i < n; i++) { //i corresponds to size of sorted array Datatype t=a[i]; //save the value you are sorting for (int j = i; j > 0 && t < a[j-1]; j--) //loop through sorted array a[j] = a[j - 1]; //push back the values that are greater a[j] = t; //put the sort value in place } } 71

Insertion Sort Internal vs. External? Stable? Oblivious? 72

Insertion Sort Internal vs. External? Internal Stable? Yes Oblivious? No 73

Insertion Sort Stable? Yes You only “swap” with adjacent values Saving the value, then pushing back greater values is the same as swapping adjacent values until it is in place Saving is just more efficient Smith, John Brown, Nick Young, Bob 74 Smith, Kim Brown, Joe

Insertion Sort Oblivious? No If it is already sorted, then the inner loop doesn’t happen Think about a sorted array, how much work gets done? 1 2 8 13 75 22 34 36

Insertion Sort Oblivious? No If it is already sorted, then the inner loop doesn’t happen Think about a sorted array, how much work gets done? N because each element is larger than the largest in the sorted array, so it stays in place 1 2 8 13 76 22 34 36

Insertion Sort Improvements: Do a binary search to find where to insert Complexity? 77

Insertion Sort Improvements: Do a binary search to find where to insert Complexity? Stays the same because you still have to move back any values that are greater 78

Insertion Sort Improvements: Store the sorted array in a linked list so that you don’t have to push back values that are greater Complexity? 79

Insertion Sort Improvements: Store the sorted array in a linked list so that you don’t have to push back values that are greater Complexity? Stays the same You cannot use binary searches in linked lists, so you still have to do n work to find where to insert 80

Shell Sort Also know as the diminishing gap sort Uses a separation gap between values it is sorting Looks at the first element, the element gap spaces away, the element 2* gap spaces away, … and sorts those values Shifts the gap Diminishes the gap 81

Shell Sort Consider gap=5 Look at the first, sixth, and eleventh values and sort them 81 94 11 96 12 35 17 82 95 28 58 41 75 15

Shell Sort Consider gap=5 Shift the gap Look at the second, seventh, and 12 th values and sort them 35 94 11 96 12 41 17 83 95 28 58 81 75 15

Shell Sort Consider gap=5 Shift the gap Look at the second, seventh, and 12 th values and sort them 35 17 11 96 12 41 75 84 95 28 58 81 94 15

Shell Sort Consider gap=5 Shift the gap Look at the third and 8 th values and sort them 35 17 11 96 12 41 75 85 15 28 58 81 94 95

Shell Sort Consider gap=5 Shift the gap Look at the fourth and 9 th values and sort them 35 17 11 28 12 41 75 86 15 96 58 81 94 95

Shell Sort Gap=3 Look at the 1 st, 4 th, 7 th, 10 th, and 13 th values and sort them 35 17 11 28 12 41 75 87 15 96 58 81 94 95

Shell Sort Gap=3 Look at the 2 nd , 5 th, 8 th, and 11 th values and sort them 28 17 11 35 12 41 58 88 15 96 75 81 94 95

Shell Sort Gap=3 Look at the 3 rd, 6 th, 9 th, and 12 th values and sort them 28 12 11 35 15 41 58 89 17 96 75 81 94 95

Shell Sort Now reduce your gap, say to 1 Look at all the values and sort them This is basically an insertion sort 28 12 11 35 15 41 58 90 17 94 75 81 96 95

Shell Sort Gap=1 Look at all the values and sort them This is basically an insertion sort 11 12 15 17 28 35 41 91 58 75 81 94 95 96

Shell Sort Avoid large amounts of data movement by: Comparing elements that are far apart. Comparing elements that are less far apart. Gradually shrinks toward insertion sort. 92

Shell Sort A sub-quadratic algorithm whose code is only slightly longer than insertion sort, making it the simplest of the faster algorithms. 93

Shell Sort Code void Shell. Sort (Data. Type* a, int length ) { for(int gap=length/2; gap>0; gap == 2 ? 1 : (int)gap / 2. 2 ) ) for ( int i = gap; i < length; i++ ){ Data. Type tmp = a[i]; for ( ; j >= gap && tmp < a[j-gap]; j -= gap ) a[j] = a[j-gap]; a[j] = tmp; } } 94

Shell Sort Complexity Very complicated analysis Worst case running time is Ο(n*n) Average run time is between O(n^(7/6)) and O(n^(5/4)) Can get bad running times if all big numbers are at even positions and all small numbers are at odd AND the gaps don’t rearrange them as they are always even. 95

Shell Sort Internal vs. External? Stable? Oblivious? 96

Shell Sort Internal vs. External? Internal Stable? No, swapping across the gaps Oblivious? Supposedly no because it builds off insertion 97

Heap Sort Use a heap to sort data Heaps have no real order, but they do keep track of the min/max How does this work? 98

Heap Sort Use a heap to sort data Heaps have no real order, but they do keep track of the min/max How does this work? Can keep popping the min, and placing in the next position of the array 99

Heap Sort Use a heap to sort data Algorithm Insert each item into a heap Delete min, and place in the array Continue deleting until the heap is empty 100

Heap Sort Use a heap to sort data Insert each item into a heap 13 34 1 8 36 101 2 22

Heap Sort Use a heap to sort data Insert each item into a heap 13 34 1 8 36 102 2 22

Heap Sort Use a heap to sort data Delete min, and place in the array 13 34 1 8 36 103 2 22

Heap Sort Use a heap to sort data Continue deleting until the heap is empty 1 34 1 8 36 104 2 22

Heap Sort Use a heap to sort data Continue deleting until the heap is empty 1 2 1 8 36 105 2 22

Heap Sort Use a heap to sort data Continue deleting until the heap is empty 1 2 8 8 36 106 2 22

Heap Sort Use a heap to sort data Continue deleting until the heap is empty 1 2 8 13 107 36 2 22

Heap Sort Use a heap to sort data Continue deleting until the heap is empty 1 2 8 13 108 22 2 22

Heap Sort Use a heap to sort data Continue deleting until the heap is empty 1 2 8 13 109 22 34 22

Heap Sort Use a heap to sort data Continue deleting until the heap is empty Now the array is sorted! 1 2 8 13 110 22 34 36

Heap Sort Complexity? 111

Heap Sort Complexity? Building the heap Insertion into a heap is logn This is done n times Deleting from the heap Deleting from a heap is logn This is done n times nlogn + nlogn O(nlogn) 112

Heap Sort Code void Heap. Sort(Datatype* a, int n) { Min. Heap my. Heap; for (int i = 0; i < n; i++) my. Heap. insert(a[i]); for(int i=0; i<n; i++) a[i] = my. Heap. delete. Min() } 113

Heap Sort Internal vs. External? Stable? Oblivious? 114

Heap Sort Internal vs. External? Internal Stable? No, heap operations mix up values Oblivious? Sort of, if the array is ordered, the heap insert operations will be constant, but the delete operations are still logn 115

Divide and Conquer Algorithms Divide: Divide the input into 2+ sets. This reduces the size and makes each set more manageable than the original. Recurse: Typically use recursion to solve the subproblems created by dividing. Conquer: Combine the solutions from the subproblems into the overall solution. 116

Divide and Conquer Algorithms In sorting, this method of algorithm design can create very efficient sorts. Merge. Sort and Quick. Sort are both divide and conquer algorithms. 117

Merge Sort The Algorithm: Divide the array in half Sort the halves Merge them together 118

Merge Sort The Algorithm: Divide the array in half Sort the halves Do this recursively Have your base case return when the array has size=1 Merge them together Keep track of the “beginning” of both arrays, and “pop” the smaller of the two values. Place it in your result array. 119

Merge Sort Divide the array in half 13 34 1 8 36 120 2 22 9

Merge Sort Recursively solve the two halves Divide the array in half 13 34 1 8 36 121 2 22 9

Merge Sort Recursively solve the two halves Divide the array in half 13 34 1 8 36 122 2 22 9

Merge Sort Recursively solve the two halves Now we have hit the base case, and an array of one is sorted We will go back up combining in order 13 34 1 8 36 123 2 22 9

Merge Sort Recursively solve the two halves We will go back up combining in order 13 34 1 8 2 124 36 9 22

Merge Sort Recursively solve the two halves We will go back up combining in order 1 8 13 34 2 125 9 22 36

Merge Sort Recursively solve the two halves The array is sorted! 1 2 8 9 13 126 22 34 36

Merge Sort Complexity? Levels of Recursion? Work done at each level? 127

Merge Sort Complexity? Levels of Recursion? Logn depth, because the array is divided in half each time Work done at each level? N work done in recombining the arrays O(n* logn) 128

Merge Sort Merge sort requires extra memory Every time the array is divided in half, you create two arrays to hold the halves Memory use isn’t n like the other, in place, sorts Memory use is n logn If you are really clever, you can do it in 2*n 129

Merge Sort Code void Merge. Sort(int* a, int size){ if(size==1) return; int* half 1 = new int[size/2], half 2=new int[size/2]; copy. Data(a, size, half 1, half 2); Merge. Sort(half 1, size/2); Merge. Sort(half 2, size/2); int i=0, h 1=0, h 2=0; while ((h 1 <size/2) || (h 2 <size/2)) if ((half 1[h 1] <= half 2[h 2] && h 1<size/2) || h 2==size/2) a[i++] = half 1[h 1++]; else } a[i++] = half 2[h 2++]; 130

Merge Sort Internal vs. External? Stable? Oblivious? 131

Merge Sort Internal vs. External? Rarely done internally because of memory overhead. Used for External sorts, because chunks can be brought into memory and sorted. Unlike other sorts, this never needs to see the entire array to sort it. Stable? Yes, the only time things change order is in merging, and you control that Oblivious? Yes, it will do the same amount of work with a sorted array 132