CSE 221ICT 221 Analysis and Design of Algorithms

CSE 221/ICT 221 Analysis and Design of Algorithms Lecture 05: Analysis of time Complexity of Sorting Algorithms Dr. Surasak Mungsing E-mail: Surasak. mu@spu. ac. th Sep-20 1

Sorting Algorithms § § § § § Bin Sort Radix Sort Insertion Sort Shell Sort Selection Sort Heap Sort Bubble Sort Quick Sort Merge Sort 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 2

9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 3

Bin Sort A 2 B 4 C 5 D 4 E 3 F 0 G 4 H 3 A J H E I G D B C Bin 2 Bin 3 Bin 4 Bin 5 D 4 G 4 I 4 J 3 (a) Input chain F Bin 0 Bin 1 (b) Nodes in bins F 0 A 2 E 3 H 3 J 3 B 4 I 4 C 5 (c) Sorted chain 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 4

Radix Sort with r=10 and d=3 216 521 425 116 91 515 124 34 96 24 515 216 116 96 (a) Input chain 521 91 124 34 24 425 (b) Chain after sorting on least significant digit 515 216 116 521 124 24 425 34 91 96 (c) Chain after sorting on second-least significant digit 24 34 91 96 116 124 216 425 (d) Chain after sorting on most significant digit 9/18/2020 5 515 521

Insertion Sort Concept 9/18/2020 6

9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 7

Shell Sort Algorithm 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 8

Shell Sort Algorithm 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 9

Shell Sort Algorithm 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 10

9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 11

Shell Sort Demo http//: e-learning. mfu. ac. th/mflu/1302251/demo/Chap 03/Shell. Sort. html 9/18/2020 12

Selection Sort Concept 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 13

9/18/2020 14

Heap Sort Algorithm 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 15

9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 16

Bubble Sort Concept 9/18/2020 17

9/18/2020 18

Quick sort The fastest known sorting algorithm in practice. 81 13 43 92 31 57 75 26 65 0 Select pivot 81 13 13 9/18/2020 31 57 26 43 0 43 92 31 57 65 75 26 0 Partition 65 92 75 81 CSE 221/ICT 221 Analysis and Design of Algorithms 19

Quick sort 13 31 57 26 43 0 65 Quick sort small 0 13 26 31 43 57 81 Quick sort large 75 65 0 13 26 31 43 57 65 9/18/2020 75 92 75 81 81 CSE 221/ICT 221 Analysis and Design of Algorithms 92 92 20

Quick sort 9/18/2020 21

Quick Sort Partitions 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 22

Quick sort 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 23

External Sort: A simple merge 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 24

Merge Sort 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 25

Merge Sort 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 26

9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 27

9/18/2020 28

Analysis of Insertion Sort for (int i = 1; i < a. length; i++) {// insert a[i] into a[0: i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; } 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 29

Complexity § Space/Memory § Time § Count a particular operation § Count number of steps § Asymptotic complexity 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 30

Comparison Count for (int i = 1; i < a. length; i++) {// insert a[i] into a[0: i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; } 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 31

Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; How many comparisons are made? 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 32

Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; number of compares depends on a[]s and t as well as on i 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 33

Comparison Count Ø Worst-case count = maximum count Ø Best-case count = minimum count Ø Average count 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 34

Worst-Case Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a = [1, 2, 3, 4] and t = 0 4 compares a = [1, 2, 3, …, i] and t = 0 i compares 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 35

Worst-Case Comparison Count for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j --) a[j + 1] = a[j]; total compares = 1 + 2 + 3 + … + (n-1) n T(n) = i=2 (i-1) = (n-1)n/2 = O(n 2) 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 36

Average-Case Comparison Count for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; n T = i=2 i-1 2 = O(n 2) 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 37

Analysis of Quick Sort 81 13 43 92 31 57 75 26 65 0 Select pivot 81 13 31 57 13 26 43 0 9/18/2020 43 92 31 57 65 75 26 0 Partition 65 92 75 81 CSE 221/ICT 221 Analysis and Design of Algorithms 38

Main Quick Sort Routine 9/18/2020 private static void quicksort( Comparable [ ] a, int left, int right ) { /* 1*/ if( left + CUTOFF <= right ) { /* 2*/ Comparable pivot = median 3( a, left, right ); // Begin partitioning /* 3*/ int i = left, j = right - 1; /* 4*/ for( ; ; ) { /* 5*/ while( a[ ++i ]. compare. To( pivot ) < 0 ) { } /* 6*/ while( a[ --j ]. compare. To( pivot ) > 0 ) { } /* 7*/ if( i < j ) /* 8*/ swap. References( a, i, j ); else /* 9*/ break; } /*10*/ swap. References( a, i, right - 1 ); // Restore pivot /*11*/ quicksort( a, left, i - 1 ); // Sort small elements /*12*/ quicksort( a, i + 1, right ); // Sort large elements } else // Do an insertion sort on the subarray /*13*/ insertion. Sort( a, left, right ); } CSE 221/ICT 221 Analysis and Design of Algorithms 39

Best-Case Analysis The pivot is in the middle; T(n) = 2 T(n/2) + cn Divide both sides by n; T(n) n T(n/2) n/2 T(n/4) n/4 T(2) 2 Add all equations; T(n) n = = = … = = T(n/2) n/2 T(n/4) n/4 T(n/8) n/8 T(1) 1 + c + c + c log n T(n) = c n logn + n = O(n log n) 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 41

Average-Case Analysis (1/4) Total time; T(N) = T(i) + T(N-i-1) + c N(1). . ………… Average time of T(i) and T(N-i-1) is 1 N -1 N Therefore T(N)= 2 N T( j ) …………. . (2) j=0 N -1 T( j ) + c. N …………. . (3) T( j ) + c. N 2 …………. . (4) + c(N – 1)2 …………. . (5) j=0 N -1 NT(N)= 2 j=0 N -2 (N-1) T(N-1) = 2 j=0 9/18/2020 T( j ) CSE 221/ICT 221 Analysis and Design of Algorithms 42

Average-Case Analysis(2/4) (5); (4) – NT(N) – (N-1) T(N-1) = 2 T(N-1) +2 c. N - c …………. . (6) Rearrange terms in equation and ignore c on the right-hand side; NT(N) = (N+1) T(N-1) +2 c. N (7) divides by N(N+1); T(N) N+1 9/18/2020 = T(N-1) N + 2 c …………. . (7) …………. . (8) N+1 CSE 221/ICT 221 Analysis and Design of Algorithms 43

Average-Case Analysis(3/4) T(N) …………. . (8) N N+1 T(N-1) T(N-2) 2 c = + …………. . (9) N N-1 N T(N-2) T(N-3) 2 c T(2) =. . . = N-1 T(1) 2 c 2 T(N) T(1) = + N-2 3 N+1 9/18/2020 + 2 c N+1 N-1 Sun equations (8) to (11); = T(N-1) 2 + …………. . (10) …………. . (11) 3 N +1 + 2 c CSE 221/ICT 221 Analysis and Design of Algorithms …………. . (12) i=3 44

Average-Case Analysis(4/4) T(N) = T(1) N +1 + 2 c 2 N+1 …………. . (12) i=3 Sum in equation (12) ia approximately log. C(N+1)+ - 3/2, which is Euler’s constant 0. 577 therefore and T(N) N+1 = O(log N( T(N) = O(Nlog N( …………. . (13) …………. . (14) In summary, time complexity of Quick sort algorithm for Average-Case is T(n) = O(n log n) 9/18/2020 CSE 221/ICT 221 Analysis and Design of Algorithms 45

18 -Sep-20 46