So far so good but can we do

So far so good, but can we do better? Yes, cheaper by halves. . . http: //www. cs. miami. edu/~burt/learning/Csc 517. 101/w orkbook/cheaperbyhalf. html Let’s try it out … CSC 317 1

Toward merge sort CSC 317 2

What is it going to cost us? • Divide Left and Right: constant C 1 • Sort Left with Insertion Sort: • Sort. Right: with. Insertion. Sort • Merge: C 3 n • Total: (Have we gained from Insertion sort (Cn 2)? CSC 317 3

Another example: find min Input: array A Output: minimum value in array A Cost? C 1 n CSC 317 4

Is splitting the array in half going to help us? Let’s see: • Divide Left and Right: constant C 1 • Find min Left: • Find min Right: • Merge: C 3 n • Total: Have we gained from find min on full array (Cn)? CSC 317 5

Back to merge sort: If we can split once, can we do it more often? CSC 317 6

Merge sort: pseudo code (step-by-step) Merge-Sort If array larger than size 1 Divide array into Left and Right arrays // divide Merge-Sort(Left array) // conquer left; recursive call Merge-Sort(Right array) // conquer right; recursive call Merge sorted Left and Right arrays // combine CSC 317 7

Merge sort: pseudo code (more formal) length of array A // divide (split in half) // conquer left // conquer right // combine At what step do we have most work? Merging step, otherwise we just split arrays CSC 317 8

Merge sort: run time analysis Total work in each step: • Divide: constant • Combine: Cn • Conquer: recursively solve two subproblems, each size n/2 We’ll write out the recursion as recursion combination grows like n log 2 n • Compared to insertion sort is this good or bad? • And how did we end up here? CSC 317 9

What is this? A recursion tree where recursion combination is Cn CSC 317 10

What is this? A recursion tree where recursion combination is Cn CSC 317 11

Next step in the recursion: Cn Cn/2 CSC 317 12

Let’s keep on going (I mean splitting)! Cn Cn/2 Each row adds to how much work? Cn/4 C CC CSC 317 C C C 13

C Cn Cn Cn/2 Cn Cn/4 Cn C C C C CSC 317 Cost per level stays the same! Cn 14

C Cn Level 0: Cn/2 Level 1: Cn/4 Level 2: C C C C CSC 317 Level k: 15

We know that n = 1 and We need to find level k (height of tree) number of levels, each level needs work Cn Total work: Cn lg 2(n) CONCLUSION: • grows as n lg 2(n) • Insert sort: n 2 CSC 317 Is mergesort always faster? Any disadvantages? 16

Correctness and loop invariants • How do we know that an algorithm is correct (i. e. always gives the right answer? • We use loop invariants (what does that mean? ). • Invariant = something that does not change • Loop invariant = a property about the algorithm that does not change at every iteration before the loop • Usually the property we would like to prove is correct about the algorithm! • Intuitive, but we would like to state mathematically CSC 317 17

Loop invariant example: insertion sort Question: What invariant property makes this algorithm correct? CSC 317 18

Loop invariant example: insertion sort KEY KEY KEY Question: What invariant property makes this algorithm correct? Answer: Before each iteration of the for loop, the elements thus far are sorted. Next question: Can we state that mathematically? CSC 317 19

Loop invariant example: insertion sort KEY KEY KEY Insertion sort loop invariant: at the start of each iteration of the for loop, A[1, . . . , j-1] consists of elements originally in A[1, . . . , j-1] but in sorted order. CSC 317 20

Can we prove this? 3 steps: • Initialization: Algorithm is true prior to first iteration of the loop (base case). • Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). • Termination: When the loop terminates, the invariant gives a useful property that shows the algorithm is correct. CSC 317 21

• Initialization: Algorithm is true prior to first iteration of the loop (base case). When j=2, A[1] is just one element, which is the original element in A[1], and must be already sorted Insert sort pseudocode j=2 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1, . . . , j-1] by comparing and swapping into correct position CSC 317 22

• Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY KEY KEY true for j-1 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1, . . . , j-1] by comparing and swapping into correct position Might not be true in loop. But make sure here that it remains sorted, by pairwise swaps. CSC 317 23

• Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY KEY KEY So will remain true for j 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1, . . . , j-1] by comparing and swapping into correct position We make sure here that it remains sorted, by pairwise swaps CSC 317 24

• Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY KEY KEY • If A[1, . . . , j-1] sorted before iteration of loop, then for key=A[j], we pairwise swap it into correct position; so now A[1, . . . , j] is also sorted. • Also, A[1, . . . , j-1] includes elements originally in A[1, . . . , j-1]. Then A[1, . . . , j] includes those elements and the element A[j], so must include elements originally in A[1, . . . , j] CSC 317 25

• Termination: When the loop terminates, the invariant gives a useful property that shows the algorithm is correct. When the loop terminates (i. e. j=n+1) A[1, . . . , j] must be in sorted order, which is A[1, …, n] or the entire array. true for j= n + 1 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1, . . . , j-1] by comparing and swapping into correct position CSC 317 26

Loop invariants example: find min Animation example (Burt Rosenberg) http: //www. cs. miami. edu/~burt/learning/Csc 517. 101/workbook/findmin. html Input: array A Output: minimum value in array A Question: What invariant would make this algorithm correct? CSC 317 27

Input: array A Output: minimum value in array A Question: What invariant would make this algorithm correct? Answer: At each point of the algorithm, the current min is known. More formally … CSC 317 28

Input: array A Output: minimum value in array A Loop invariant: At each iteration of the for loop, min is the smallest Element in A[1, …, i-1]. CSC 317 29

k Loop invariant of merge: i Loop invariant: At the start of each iteration of the for loop, A[p, . . . , k-1] contains the k-p smallest elements, in sorted order. Also, L[i] and R[j] are the smallest elements of their arrays not yet copied back into A. J j i k CSC 317 30