Analysis of Algorithms CS 105 Introduction to Data

Analysis of Algorithms CS 105 Introduction to Data Structures and Algorithms 1

What is a good algorithm? n n It must be correct It must be efficient n Implementations of an algorithm must run as fast as possible How is this measured? n Running time n 2

Running time of a program n n Amount of time required for a program to execute for a given input If measured experimentally, n n Dependent on hardware, operating system and software Answer will be in milliseconds, minutes, hours, … 3

Running time of an algorithm n n n Answer will not be in seconds or minutes Instead, we count the number of operations carried out Result will be a formula in terms of some input size, n n n Takes into account all possible inputs Example statement on running time: Running time of algorithm A is T(n) = 2 n + 3 4

Describing algorithms n n n First, we need to agree on how to describe or specify an algorithm Note: algorithms are intended for humans (programs are intended for computers) Descriptions should be high-level explanations that combine natural language and familiar programming structures: Pseudo-code 5

Pseudo-code example Algorithm array. Max(A, n): Input: An array A storing n integers. Output: The maximum element in A. current. Max A[0] for i 1 to n - 1 do if current. Max < A[i] then current. Max A[i] return current. Max 6

Pseudo-Code conventions n n General Algorithm Structure Statements Expressions Control Structures 7

Algorithm Structure Algorithm heading Algorithm name(param 1, param 2, . . . ): Input : input elements Output : output elements statements… 8

Statements n n Assignment: use instead of = Method or function call: n n n Return statement: n n object. method(arguments) return expression Control structures 9

Control Structures n n decision structures while loops repeat loops n if. . . then. . . [else. . . ] while. . . do n repeat. . . until. . . n for. . . do n for loop 10

Expressions n Standard math symbols + n - * / () Relational operators = > < <= >= != n Boolean operators and n n or not Assignment operator Array indexing A[i] 11

General rules on pseudo-code n Should communicate high-level ideas and not implementation details n n Syntax is not as tight (e. g. , indentation and line-breaks take the place of ; and {} in Java) Clear and informative 12

Back to running time n n Once an algorithm has been described in pseudo-code, we can now count the primitive operations carried out by the algorithm Primitive operations: assignment, calls, arithmetic operations, comparisons, array accesses, return statements, … 13

Back to array. Max example n n n Suppose the input array is A = {42, 6, 88, 53}, (n = 4) How many primitive operations are carried out? Some notes n n The for statement implies assignments, comparisons, subtractions, and increments the statement current. Max A[i] will sometimes not be carried out 14

What to count current. Max A[0]: assignment, array access for i 1 to n - 1 do: assignment, comparison, subtraction, increment (2 ops) if current. Max < A[i] then: comparison, access current. Max A[i]: assignment, access return current. Max: return 15

Counting operations current. Max A[0] for i 1 to n - 1 do if current. Max < A[i] then current. Max A[i] return current. Max 2 15 6 2 1 OPERATIONS CARRIED OUT 26 16

Handling all possible inputs n n In the example just carried out, running time = 26 for the given input A more useful answer is to provide a formula that applies in general n n Formula is in terms of n, the input or array size Since there may be statements that do not always execute, depending on input values, there are two approaches n n Assume worst-case Provide a range 17

Measuring running time comparison & subtraction increment current. Max A[0] for i 1 to n - 1 do if current. Max < A[i] then current. Max A[i] return current. Max 2 1+2 n+2(n-1) 0. . 2(n-1) 1 RUNNING TIME (worst-case) 8 n - 2 18

Nested loops // prints all pairs from the set {1, 2, …n} for i 1 to n do for j 1 to n do print i, j How many times does the print statement execute? 19

Nested loops // prints all pairs from the set {1, 2, …n} for i 1 to n do for j 1 to n do print i, j n 2 times How about the operations implied in the for statement headers? 20

for statement revisited for i 1 to n do … 1 assignment i 1 n+1 comparisons i <= n for each value of i in {1, 2, …, n+1} n increments = n assignments + n adds => 3 n+2 operations 21

Nested loops running time for i 1 to n do for j 1 to n do print i, j 3 n+2 n (3 n+2) n 2 TOTAL: 4 n 2 + 5 n + 2 22

Nested loops example 2 // prints all pairs from the set {1, 2, …n} // excluding redundancies (i. e. , 1, 1 is not a // real pair, 1, 2 and 2, 1 are the same pair) for i 1 to n-1 do for j i+1 to n do print i, j How many times does the print statement execute? 23

Nested loops example 2 for i 1 to n-1 do for j i+1 to n do print i, j when … when i = 1, i = 2, n-1 prints n-2 prints i = n-2, i = n-1, 2 prints 1 print 24

Nested loops example 2 Number of executions of the print statement: 1 + 2 + 3 + … + n-1 = n(n-1)/2 = ½n 2 – ½n 25

Nested loops example 2 for i 1 to n-1 do for j i+1 to n do print i, j Take-home exercise: count the operations implied by the for statement headers 26

Running time function considerations n n n In determining the running time of an algorithm, an exact formula is possible only when the operations to be counted are explicitly specified Different answers will result if we choose not to count particular operations Problem: variations in the answers appear arbitrary 27

Classifying running time functions n n n We want to classify running times under particular function categories Goal: some assessment of the running time of the algorithm that eliminates the arbitrariness of counting operations Example: array. Max runs in time proportional to n 28

Some function categories n n n n The The constant function: linear function: quadratic function: cubic function: exponential function: logarithm function: n log n function: f(n) f(n) = = = = c n n 2 n 3 bn log n 29

Big-Oh notation n Consider the following (running time) functions f 1(n) = 2 n + 3 f 2(n) = 4 n 2 + 5 n + 2 f 3(n) = 5 n n n We place f 1 and f 3 under the same category (both are proportional to n) and f 2 under a separate category (n 2 ) We say: n n 2 n+3 is O(n), and 5 n is O(n), while 4 n 2 + 5 n + 2 is O(n 2) 2 n+3 and 5 n are linear functions, while 4 n 2 + 5 n + 2 is a quadratic function 30

Significance of Big-Oh n n Example for i 0 to n-1 do A[i] 0 Running time is 2 n, if we count the assignments & array accesses in the loop body (A[i] 0) 4 n+1, if we include implicit loop assignments/adds 5 n+2, if we include implicit loop comparisons n Regardless, running time is O(n) 31

Prefix Averages Problem n n n Given an array X storing n numbers, compute prefix averages or running averages of the sequence of numbers A[i] = average of X[0], X[1], …, X[i] Example n n n Input: Output: X = {1, 3, 8, 2} A = {1, 2, 4, 3. 5} Algorithm? 32

Algorithm 1 Algorithm prefix. Averages 1(A, n): Input: An array X of n numbers. Output: An array A containing prefix averages for i 0 to n - 1 do sum 0 for j 0 to i do sum + X[j] A[i] sum/(i+1) return array A 33

Running time of Algorithm 1 n n n Exercise: count primitive operations carried out by Algorithm 1 Regardless of what you decide to count, the answer will be of the form: a n 2 + b n + c Observe that the answer is O(n 2) 34

Algorithm 2 Algorithm prefix. Averages 2(A, n): Input: An array X of n numbers. Output: An array A containing prefix averages sum 0 for i 0 to n - 1 do sum + X[i] A[i] sum/(i+1) return array A 35

Running time of Algorithm 2 n n n Running time of Algorithm 2 will be of the form: a n + b Observe that the answer is O(n) Assessment: Algorithm 2 is more efficient than Algorithm 1, particularly for large input (observe the growth rate of n and n 2) 36

Summary n n Algorithms are expressed in pseudo-code The running time of an algorithm is the number of operations it carries out in terms of input size n n n Answer depends on which operation(s) we decide to count Big-Oh notation allows us to categorize running times according to function categories Next: formal definition of Big-Oh notation 37