Algorithms Sorting The Sorting Problem Input A sequence

Algorithms Sorting

The Sorting Problem • Input: – A sequence of n numbers a 1, a 2, . . . , an • Output: – A permutation (reordering) a 1’, a 2’, . . . , an’ of the input sequence such that a 1’ ≤ a 2’ ≤ · · · ≤ an’ 2

Example 53 10 2 62 128 93 28 18 2 10 18 28 53 62 93 128

Sorting algorithms • • Insertion sort Selection sort Bubble sort Heap sort Merge sort Quick sort Bucket sort Radix sort O( n 2 ) O( n log n ) O( n )

Insertion Sort • Idea: like sorting a hand of playing cards – Start with an empty left hand the cards facing down on the table. – Remove one card at a time from the table, and insert it into the correct position in the left hand • compare it with each of the cards already in the hand, from right to left – The cards held in the left hand are sorted • these cards were originally the top cards of the pile on the table 5

Insertion Sort 6 10 24 36 To insert 12, we need to make room for it by moving first 36 and then 24. 12 6

Insertion Sort 6 10 24 36 12 7

Insertion Sort 6 10 24 3 6 12 8

Insertion Sort input array 5 2 4 6 1 3 at each iteration, the array is divided in two sub-arrays: left sub-array sorted right sub-array unsorted 9

Insertion Sort 10

Insertion sort example

An Example: Insertion Sort 30 10 40 20 1 2 3 4 i = j = key = A[j] = A[j+1] = Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 30 10 40 20 1 2 3 4 i=2 j=1 A[j] = 30 key = 10 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=1 A[j] = 30 key = 10 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=0 A[j] = key = 10 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=2 j=0 A[j] = key = 10 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=0 A[j] = key = 10 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=0 A[j] = key = 40 A[j+1] = 10 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=2 A[j] = 30 key = 40 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=2 A[j] = 30 key = 40 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 20 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 30 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 20 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 20 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }

An Example: Insertion Sort 10 20 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 20 Insertion. Sort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Done!

Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 2 3 4 8 9 6 2 3 4 6 8 9 done

Analysis of Insertion Sort INSERTION-SORT(A) cost c 1 c 2 Insert A[ j ] into the sorted sequence A[1. . j -1] 0 c 4 i←j-1 c 5 while i > 0 and A[i] > key c 6 do A[i + 1] ← A[i] c 7 i←i– 1 c 8 A[i + 1] ← key for j ← 2 to n do key ← A[ j ] times n n-1 n-1 tj: # of times the while statement is executed at iteration j 42

Best Case Analysis • The array is already sorted “while i > 0 and A[i] > key” – tj = 1 • T(n) = c 1 n + c 2(n -1) + c 4(n -1) + c 5(n -1) + c 8(n-1) = (c 1 + c 2 + c 4 + c 5 + c 8)n + (c 2 + c 4 + c 5 + c 8) = an + b = (n) 43

Worst Case Analysis • The array is in reverse sorted order “while i > 0 and A[i] > key” – Always A[i] > key in while loop test – Have to compare key with all elements to the left of the j-th position compare with j-1 elements tj = j using we have: a quadratic function of n • T(n) = (n 2) order of growth in n 2 44

Bubble Sort • Idea: – Repeatedly pass through the array – Swaps adjacent elements that are out of order i 1 2 3 8 4 6 n 9 2 3 1 j • Easier to implement, but slower than Insertion sort 45

Example 8 4 6 9 2 3 i=1 8 4 6 9 2 4 6 3 9 1 2 4 4 6 1 9 6 i=1 1 6 1 2 8 4 6 1 i=1 j 1 8 i=1 j 4 4 6 6 9 i=3 3 1 2 3 8 4 6 i=4 2 3 1 2 3 9 9 2 2 3 3 1 2 3 3 4 4 4 9 j 8 6 i=5 9 3 j 6 j 8 2 j j 4 9 i=2 j i=1 8 1 6 9 j 8 9 i=6 j 8 9 i=7 j 46

Bubble Sort Alg. : BUBBLESORT(A) for i 1 to length[A] do for j length[A] downto i + 1 do if A[j] < A[j -1] then exchange A[j] A[j-1] i 8 i=1 4 6 9 2 3 1 j 47

Bubble-Sort Running Time Alg. : BUBBLESORT(A) for i 1 to length[A] c 1 do for j length[A] downto i + 1 c 2 c 3 Comparisons: n 2/2 do if A[j] < A[j -1] then exchange A[j] A[j-1] Exchanges: n 2/2 T(n) = c 1(n+1) + c 2 c 3 c 4 = (n) + (c 2 + c 4) Thus, T(n) = (n 2) 48

Selection Sort • Idea: – Find the smallest element in the array – Exchange it with the element in the first position – Find the second smallest element and exchange it with the element in the second position – Continue until the array is sorted • Disadvantage: – Running time depends only slightly on the amount of order in the file 49

Example 8 4 6 9 2 3 1 1 2 3 4 9 6 8 1 4 6 9 2 3 8 1 2 3 4 6 9 8 1 2 6 9 4 3 8 1 2 3 4 6 8 9 1 2 3 9 4 6 8 1 2 3 4 6 8 9 50

Selection Sort Alg. : SELECTION-SORT(A) n ← length[A] 8 4 6 for j ← 1 to n - 1 do smallest ← j for i ← j + 1 to n do if A[i] < A[smallest] then smallest ← i exchange A[j] ↔ A[smallest] 9 2 3 1 51

Analysis of Selection Sort Alg. : SELECTION-SORT(A) times n ← length[A] c 1 1 for j ← 1 to n - 1 c 2 n c 3 n-1 n 2/2 do smallest ← j for i ← j + 1 to n comparisons n cost exchanges c 4 do if A[i] < A[smallest] c 5 then smallest ← i c 6 exchange A[j] ↔ A[smallest] c 7 n-1 52

Time complexity summary Algorithm Insertion sort Selection sort Bubble sort Best case O( n ) Worst case O(n 2 ) O(n 2 )