Introduction to Algorithms Insertion Sort CSE 680 Prof

Introduction to Algorithms Insertion Sort CSE 680 Prof. Roger Crawfis

Insertion Sort Overview How do we know where to place the next card? l What assumptions do we make at each step? l What is the algorithm? l

Insertion Sort Algorithm l Given: a set U of unsorted elements l Algorithm: 1. 2. 3. l Foreach element e in U remove e from U place e in the sorted sequence S at the correct location. Step 3 needs some refinement. l l What is it’s Problem Statement? What are the Data Structures?

Inserting an Element l Any ideas on how to insert element e into our sorted sequence S?

Insertion Sort Algorithm l Your book’s pseudo-code: l What are the differences? Why go backwards?

Insertion Sort Example l The following slides are from David Luebke when he taught the course at the University of Virginia: http: //www. cs. virginia. edu/~luebke/cs 332/ l Thanks David! l Unfortunately, he flipped the indices i and j from the book.

An Example: Insertion Sort Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 7 9/9/2021

An Example: Insertion Sort 30 10 40 20 1 2 3 4 i = j = key = A[j] = A[j+1] = Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 8 9/9/2021

An Example: Insertion Sort 30 10 40 20 1 2 3 4 i=2 j=1 A[j] = 30 key = 10 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 9 9/9/2021

An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=1 A[j] = 30 key = 10 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 10 9/9/2021

An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=1 A[j] = 30 key = 10 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 11 9/9/2021

An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=0 A[j] = key = 10 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 12 9/9/2021

An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=0 A[j] = key = 10 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 13 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=2 j=0 A[j] = key = 10 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 14 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=0 A[j] = key = 10 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 15 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=0 A[j] = key = 40 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 16 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=0 A[j] = key = 40 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 17 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=2 A[j] = 30 key = 40 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 18 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=2 A[j] = 30 key = 40 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 19 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=2 A[j] = 30 key = 40 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 20 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=2 A[j] = 30 key = 40 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 21 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 22 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 23 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 20 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 24 9/9/2021

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 20 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 25 9/9/2021

An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 26 9/9/2021

An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 27 9/9/2021

An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 28 9/9/2021

An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 29 9/9/2021

An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 30 9/9/2021

An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 31 9/9/2021

An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 32 9/9/2021

An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 33 9/9/2021

An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 34 9/9/2021

An Example: Insertion Sort 10 20 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 20 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 35 9/9/2021

An Example: Insertion Sort 10 20 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 20 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Done! David Luebke 36 9/9/2021

Memory Analysis l The algorithm is said to be in-place if it uses only a constant amount of memory in accomplishing its solution. l Clearly, the book’s implementation is better than mine. l Clearly, the book’s algorithm is better than mine. That is, do we consider my implementation to be a different algorithm or the same?

Correctness We can use a property known as loop-invariance to reason about the correctness of the algorithm. l For Insertion-Sort, we can say that: l l l The subarray A[1. . i-1] elements are in sorted order A[1. . i-1] is the set of numbers from the original A[1. . i-1]

Insertion Sort Example

Loop Invariant l Initialization l True at the beginning of the loop. l Termination The loop terminates. l True, when the loop exists. l l Maintenance True at the end of each iteration of the loop. l This allows us to prove the invariant similarly to proof-by-induction. l

Insertion Sort What is the precondition Insertion. Sort(A, n) { for this loop? for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } David Luebke 41 9/9/2021

Bubble-Sort l Can you come up with loop invariants? Aka, can you prove it is correct? l How expensive is it? l When is it most expensive? l When is it least expensive? l

Selection Sort l What about selection sort? void selection. Sort(int[] a) { for (int i = 0; i < a. length - 1; i++) { int min = i; for (int j = i + 1; j < a. length; j++) { if (a[j] < a[min]) { min = j; } } if (i != min) { int swap = a[i]; a[i] = a[min]; a[min] = swap; } } }