Algorithm Analysis Lectures 3 4 Resources Data Structures

Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen, Leiserson, & Rivest): Chap. 1 Algorithms Theory & Practice (Brassard & Bratley): Chap. 1

Algorithms • An algorithm is a well-defined computational procedure that takes some value or a set of values, as input and produces some value, or a set of values as output. • Or, an algorithm is a well-specified set of instructions to be solve a problem.

Efficiency of Algorithms • Empirical – Programming competing algorithms and trying them on different instances • Theoretical – Determining mathematically the quantity of resources (execution time, memory space, etc) needed by each algorithm

Analyzing Algorithms • Predicting the resources that the algorithm requires: • Computational running time • Memory usage • Communication bandwidth • The running time of an algorithm • Number of primitive operations on a particular input size • Depends on – Input size (e. g. 60 elements vs. 70000) – The input itself ( partially sorted input for a sorting algorithm)

Order of Growth • The order (rate) of growth of a running time – Ignore machine dependant constants – Look at growth of T(n) as n – notation • Drop low-order terms • Ignore leading constants • E. g. – 3 n 3 + 90 n 2 – 2 n +5 = (n 3)

Mathematical Background

Mathematical Background • Definitions: – T(N) = O(f(N)) iff c and n 0 T(N) c. f(N) when N n 0 – T(N) = (g(N)) iff c and n 0 T(N) c. g(N) when N n 0 – T(N) = (h(N)) iff T(N) = O(h(N)) and T(N) = (h(N))

Mathematical Background • Definitions: – T(N) = o(f(N)) iff c and n 0 T(N) c. f(N) when N n 0 – T(N) = (g(N)) iff c and n 0 T(N) c. g(N) when N n 0

Mathematical Background • Rules: – If T 1(N) = O(f(N)) and T 2(N) = O(g(N)) then a) T 1(N) + T 2(N) = max( O(f(N)), O(g(N)) b) T 1(N) * T 2(N) = O(f(N) * g(N)) – If T(N) is a polynomial of degree k, then T(N) = (Nk) – Logk N = O(N) for any constant k.

More … 1. 2. 3. 4. 5. 6. 3 n 3 + 90 n 2 – 2 n +5 = O(n 3 ) 2 n 2 + 3 n +1000000 = (n 2) 2 n = o(n 2) ( set membership) 3 n 2 = O(n 2) tighter (n 2) n log n = O(n 2) True or false: 1. 2. 3. 4. n 2 = O(n 3 ) n 3 = O(n 2) 2 n+1= O(2 n) (n+1)! = O(n!)

Ranking by Order of Growth 1 n (3/2)n n log n n 2 nk 2 n (n)! (n+1)!

Running time calculations 1. Rule 1 – For Loops The running time of a for loop is at most the running time of the statement inside the for loop (including tests) times the number of iterations 2. Rule 2 – Nested Loops Analyze these inside out. The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes of all the loops

Running time calculations: Examples Example 1: sum = 0; for (i=1; i <=n; i++) sum += n; Example 2: sum = 0; for (j=1; j<=n; j++) for (i=1; i<=j; i++) sum++; for (k=0; k<n; k++) A[k] = k;

How to rank?