Complexity Analysis Examples 2 Recap Complexity analysis counts

Complexity Analysis: Examples

2 Recap § Complexity analysis counts algorithm steps Q: What is a step ? A: Something independent of the input size (n) e. g. , a multiplication, an addition, a swap a (recursive) function call

3 Simple Example: Computing An Brute force method P = 1 for (i = 1; i < n; i++) P = P * A; Complexity here is in terms of number of multiplications. It’s O(n).

4 Computing An A recursive method ? Consider calculating A 14: A 14 = A 7 x A 7, A 7 = A 3 x A, A 3 = A x A. Only 5 multiplications, not 14!

5 Computing An function raise(int A, int n) if (n == 0) return 1; if (n == 1) return A; temp = raise(A, n/2); if (n mod 2 == 0) return temp * temp; else return temp * A;

6 Useful Fact § The number of times n can be divided by 2 before reaching 0 or 1 is log 2(n)

7 Computing An function raise(int A, int n) if (n == 0) return 1; if (n == 1) return A; temp = raise(A, n/2); if (n mod 2 == 0) return temp * temp; else return temp * A; Complexity is determined by number of calls to raise. It’s the number of times n can be divided by 2 before reaching 0 or 1. So the complexity is O(log 2 n) = O(log n).

8 Another Example: Binary Search int binary_search(int n, int *a, int key) { int lo = -1; int hi = n; while (hi - lo != 1) { int mid = (hi + lo) / 2; if (a[mid] <= key) { lo = mid; } else { hi = mid; } } return lo; }

9 Another Example: Binary Search int binary_search(int n, int *a, int key) { int lo = -1; int hi = n; while (hi - lo != 1) { int mid = (hi + lo) / 2; if (a[mid] <= key) { lo = mid; } else { hi = mid; } } We half the array and we do one comparison (step) each time It’s the number of times n can be divided by 2 … : O(log 2 n)

10 Methods Method 1: Guess solution or do the computations Method 2: Iterate T(n) recurrence

11 Binary Search: Recurrence Equation T(n) = T(n/2) + 1 T(1) = 1 T(n) = 1 + T(n/2) = 1 + T(n/4) = = 1 + 1 + T(n/8) = = 1 + 1 + …. T(n/n) = log(n)

12 More Recurrence Equations int recursive(int n) { if (n <= 1) { return 1; } else { return recursive(n - 1) + recursive(n - 1); } } T(n) = 2 T(n - 1) T(1) = 1

13 Recurrence Equations int recursive(int n) { if (n <= 1) { return 1; } else { return recursive(n - 1) + recursive(n - 1); } } T(n) = 2*2*T(n-2) = 2*2*2*T(n-3) = … = 2 n T(n - n) = 2 n

14 Recurrence Equations Merge. Sort T(n) - time for merge sort = ? Calls itself recursively on the two halves, so 2 T(n/2) Merges the two sorted halves (n steps) T(n) = 2 T(n/2) + n T(1) = 1

15 Recurrence Equations T(n) – time for merge sort T(n) = n + 2 T(n/2) = n + 2 (n/2 + 2 T(n/4)) = = n + 4 T(n/4) = n + 4(n/4 + 2 T(n/8)) = = n + n + 8(n/8 + 2 T(n/16)) = … = n + n + …. base case 2 logn T(n/2 logn) = n log(n) Assume n is a power of 2

16 Methods Method 1: Guess solution or do the computations Method 2: Iterate T(n) recurrence Method 3: Recursion trees

17 Recurrence Trees: Method 3 int fibonacci (int n) { if (n <= 1) return 1; return fibonacci(n – 1) + fibonacci (n - 2); } 1 1 2 3 5 8 13 …

18 Best, average, worst case: Bubble. Sort for (i = n; i >= 1; i--) { for (j = 1; j < i; j++) { if (a[j-1] > a[j]) { temp = a[j-1]; a[j-1] = a[j]; a[j] = temp; } } } /* Line 1 */ /* 2 */ /* 3 */ /* 4 */ /* 5 */ /* 6 */ Line 1: Executed n times. Lines 2 & 3: Executed n + (n-1) + (n-2) + … + 2 + 1 = n(n + 1) / 2 times. Lines 4, 5 & 6: Executed between 0 and n(n + 1) / 2 times depending on how often a[ j-1 ] > a[ j ].

19 Best, average, worst case Let p be the probability of a[ j-1 ] > a[ j ]. In the worst case (i. e. maximum T(n)), p = 1 for all j. In the best case (i. e. minimum T(n)), p = 0 for all j. On average, p = 0. 5.

20 Sequential Search int seq_search(int n, int *a, int key) { int i = 0; while (i < n && a[i] != key) { i++; } return i; } What’s the complexity?

21 More For loops void matmpy(int n, int **c, int **a, int **b) { int i, j, k; for (i = 0; i < n; i++) { for (j = 0; j < n; j++) { c[i][j] = 0; for (k = 0; k < n; k++) { c[i][j] += a[i][k] * b[k][j]; } } What’s the complexity?

22 More For Loops /* Exercise 1 */ void mystery(int n) { int i, j, k; for (i = 1; i <= n - 1; i++) { for (j = i + 1; j <= n; j++) { for (k = 1; k <= j; k++) { /* Some statement taking O(1) time */ } }

23 More For Loops /* Exercise 2 */ void veryodd(int n) { int i, j, x, y; x = 0; y = 0; for (i = 1; i <= n; i++) { if (i % 2 == 1) { for (j = i; j <= n; j++) { x = x + 1; } for (j = 1; j <= i; j++) { y = y + 1; } }

24 Exercising loops sum = 0; for (i = 0; i < n; i++) { for (j = 0; j < i; j++) { sum++; What’s the complexity?

25 Exercising loops sum = 0; for (i = 0; i < n; i++) { for (j = 0; j < i; j++) { sum++; What’s the complexity? Si =0 to n i = n(n+1)/2

26 Exercising loops sum = 0; for (i = 0; i < n; i*=2) { for (j = 0; j < i; j++) { sum++; What’s the complexity?

27 Exercising loops sum = 0; for (i = 0; i < n; i*=2) { //Do log(n) times for (j = 0; j < n; j++) { //Do n times sum++; Assume n is a power of 2 Si =0 to logn n = n log(n)

28 Exercising loops sum = 0; for (i = 0; i < n; i*=2) { for (j = 0; j < k; j++) { sum++; What’s the complexity? Assume n is a power of 2

29 Exercising loops sum = 0; for (i = 0; i < n; i*=2) { //Do log(n) times for (j = 0; j < k; j++) { //Do k times sum++; Assume n is a power of 2 Si =0 to logn 2 i = 2 logn+1 – 1 = 2 n -1