Analysis Design of Algorithms CSCE 321 Prof Amr

Analysis & Design of Algorithms (CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 3. Time Complexity Calculations Prof. Amr Goneid, AUC 1

Time Complexity Calculations Prof. Amr Goneid, AUC 2

Time Complexity Calculations n Performance Measurement & Modeling n Performance Measurement n Performance Analysis (modeling) n Evaluating Number of Operations T(n) n Examples of Algorithm Analysis Prof. Amr Goneid, AUC 3

1. Performance Measurement and Modeling Algorithm Actual Code Actual or Pseudo code Measurement Runs Math. Model T(n) and Bounds Prof. Amr Goneid, AUC 4

2. Performance Measurement We measure the performance of an algorithm by time measurement or by counting number of operations. n Usually called Amortized Analysis: Repeat measuring the running time m times then divided by m. Important issues are: n Choice of Bound n Clocking or counting domains n Choice of Data Sizes n Choice of Data Set for the Bound Prof. Amr Goneid, AUC 5

Time Measurement Example Set repetition count m; Set problem size n; tacc = 0; Repeat on m { start time t 1 = Get. Time(); Invoke A(n); end time t 2 = Get. Time(); tacc = tacc + (t 2 - t 1); } tav = tacc / m; Prof. Amr Goneid, AUC 6

Counting Example 1 int count = 0; float s = 0. 0; float A[n][n]; for (i=1; i<=n; i++) for (j = 1, j<=n; j++) { s = s + A[i][j]; count++; } The above code finds the no. of floating point additions T(n) = count as a function of n. Prof. Amr Goneid, AUC 7

Counting Example 2: Insertion Sort Function void insertion (int a[ ], int n) { int i , v , j ; for (i =1; i < n; i++) { v = a[i]; j = i; while(j > 0 && a[j-1] > v) { a[j] = a[j-1]; j--; count++; }; a[j] = v; } } Prof. Amr Goneid, AUC 8

Insertion Sort (Performance) T 1 ordered, T 2 Random, T 3 Inversely ordered n 5 10 20 50 100 200 T 1(n) 0 0 0 T 2(n) 3 18 78 586 2609 10, 052 T 3(n) 10 45 190 1225 4950 19, 900 n(n-1)/2 10 45 190 1225 4950 19, 900 Prof. Amr Goneid, AUC 9

T(n) vs n for insertsort Prof. Amr Goneid, AUC 10

3. Performance Analysis (Modeling) n Use a high-level Description (e. g n n n Pseudocode) instead of implementation Identify problem size (n) Identify elementary processes. Assign a number of operations to each. Form a mathematical model of the sum of operations in these processes. Solve to find T(n) independent of HW Prof. Amr Goneid, AUC 11

Performance Analysis (Modeling) Example: Find the sum of all elements in a 2 -D array of size (nxn) sum ← 0. 0; for (i = 0 to n-1) for (j = 0 to n-1) sum ← sum + aij ∑i ∑j 1 The no. of floating point additions T(n) as a function of n is: Prof. Amr Goneid, AUC 12

4. Evaluating No. of Operations T(n) n Usually we specify a certain type of operations, e. g. , additions, array comparisons, etc. n Simple Statements: T(n) = no. of specified operations. e. g. for z = 2*x + y T(n) = 2 arithmetic operations n Code blocks: T(n) = Sum of sub-block T(n)’s Prof. Amr Goneid, AUC 13

No. of Operations T(n) Selection Statements: c = condition, s = statement block n if (c) s; T(n) = Tc(n) for c false (best case) = Tc(n) + Ts(n) for c true (worst case) n If (c) s 1; else s 2; Worst case T(n) = Tc(n) + max(Ts 1(n), Ts 2(n)) Prof. Amr Goneid, AUC 14

No. of Operations T(n) Example: T(n) = no. of comparisons Given that fun 1(n) does n+2 comparisons, module 1 does n 2 and module 2 does n 4. Find worst case T(n) for the segment: if ( c == fun 1(n)) call module 1; else call module 2; Process T(n) == 1 fun 1(n) n+2 module 1 n 2 module 2 n 4 Worst case total ? Prof. Amr Goneid, AUC 15

No. of Operations T(n) Repetition Statements: n For Loop Example: Process T(n) Compute m 1 T(n) = No. of multiplications Fun(i, n) does 2 i+n multiplications Total 2 i + n T(n) = ? m = n*n for (i =1; i <= m; i++) Fun(i, n); Prof. Amr Goneid, AUC 16

No. of Operations T(n) Repetition Statements: n while Loop Example: In the following code: Fun(i, n) does 3 i + log n comparisons, find T(n) = no. of comparisons i = 0; while(i <= n) {Fun(i, n); i++; } Prof. Amr Goneid, AUC 17

Running Time n The number of operations can be used to predict the running time of an algorithm for a given problem size (n) n Example: The number of operations of a sorting algorithm is directly proportional to (n log n). Direct time measurement gives 1 ms to sort 1000 items. Find how long it will take to sort 1, 000 items. Solution: Prof. Amr Goneid, AUC 18

5. Examples of Algorithm Analysis (a) Uniqueness Test Check whether all the elements in a given array are distinct n Input: An array A[0…n-1] n Output: Return “true” if all the elements in A are distinct and “false” otherwise Prof. Amr Goneid, AUC 19

Uniqueness Test Prof. Amr Goneid, AUC 20

Exercise Prove the formula: either by mathematical induction or by following the insight of a 10 year-old school boy named Carl Friedrich Gauss (1777– 1855) who grew up to become one of the greatest mathematicians of all times. Prof. Amr Goneid, AUC 21

(b) More Nested Loops n Find the number of double arithmetic operations done by the following piece of code assuming that n = 2 m: for( int t = n; t > 1; t /= 2 ) { for( int u = 1; u < n; u *= 2 ) { for( int v = 0; v < n; v += 2 ) {. . . // constant number of double arithmetic //operations } } } Prof. Amr Goneid, AUC 22

More Nested Loops The loop variables can be transformed to: Outermost loop: i = log t = m … 1, Middle loop: j = log u = 0 … m-1 innermost loop: k = v/2 = 0. . n/2 -1 Prof. Amr Goneid, AUC 23

(c) Cosine Function Evaluation The cosine function can be evaluated using a truncated infinite series: The corresponding algorithm is: float cosine (float x, int n) { float y = -x * x; float s = 1. 0; for (int i = 1; i <= n; i++) s = s + pow (y, i) / fact (2*i); return s; } Prof. Amr Goneid, AUC 24

Cosine Function Evaluation The number of arithmetic operations is evaluated as follows: n fact (2*i ) uses one mult. + 2 i mult. = 2 i+1 n pow(y, i) uses (i-1) mult. n 1 division and 1 addition inside loop n 1 mult. outside loop (-x * x) Show that: i. e. , Quadratic algorithm. Prof. Amr Goneid, AUC 25

A Faster Cosine Algorithm We express the series in the form where S(x, i) = (-x 2)i / (2 i)! , and S(x, 0) = 1 Expressing S(x, i+1) in terms of S(x, i), then where y = (-x 2) and m = 2 i with S(x, 0) = 1 Prof. Amr Goneid, AUC 26

A Faster Cosine Algorithm Show that in this case: This is a linear algorithm that is much faster than the previous quadratic algorithm Prof. Amr Goneid, AUC 27

(d) Maximum Subsequence Sum Problem Statement n Given a sequence of integers (possibly negative), a 1, a 2, . . . , an, find the maximum value of (This is zero if all integers are negative). Example: -2, 11, -4, 13, -5, -2, Smax = a 2+a 3+a 4= 20 Prof. Amr Goneid, AUC 28

Maximum Subsequence Sum Algorithm 1 Prof. Amr Goneid, AUC 29

Maximum Subsequence Sum Algorithm 1 (Analysis) Prof. Amr Goneid, AUC 30

Maximum Subsequence Sum Algorithm 2 Prof. Amr Goneid, AUC 31

Maximum Subsequence Sum Algorithm 2 (Analysis) Prof. Amr Goneid, AUC 32

Maximum Subsequence Sum Algorithm 3 Prof. Amr Goneid, AUC 33

Maximum Subsequence Sum Algorithm 3 (Analysis) Prof. Amr Goneid, AUC 34

(e) Histogram Processing A bad contrast image (F) can be enhanced to an image G by transforming every pixel intensity fij to another intensity gij using a method called Histogram Equalization Prof. Amr Goneid, AUC 35

Histogram Processing n In practice, gray levels are discrete (k = 0. . L-1) n A histogram is an array whose element nk = no. of pixels with gray level k (k=0 is black, k=L-1 is white). n Algorithm to build histogram for an image with N rows and M columns (Ntot = NM pixels): Prof. Amr Goneid, AUC 36

Histogram Equalization Algorithm (1) The histogram is used in the following algorithm: Prof. Amr Goneid, AUC 37

Histogram Equalization Algorithm (2) A better algorithm first builds a Cumulative array: Prof. Amr Goneid, AUC 38

Histogram Equalization Summary of Total Complexity Algorithm (1): Algorithm (2): Prof. Amr Goneid, AUC 39

Histogram Equalization Numerical Example L = 256 gray levels Algorithm (1): Algorithm (2): Prof. Amr Goneid, AUC 40