Discrete Mathematics Algorithms Introduction p An algorithm is

Discrete Mathematics Algorithms

Introduction p An algorithm is a finite set of instructions with the following characteristics: v Precision: steps are precisely stated v Uniqueness: Results of each step of execution are uniquely defined. They depend only on inputs and results of preceding steps v Finiteness: the algorithm stops after a finite (maybe large) number of steps for any input in the set.

More characteristics of algorithms Input: the algorithm receives input from a specified set v Output: The algorithm produces output. From each set of input, a set of output is produced. Outputs are solution to problem. v Generality: the algorithm applies to various sets of inputs v Effectiveness : It must be possible to perform each step of an algo exactly and in a finite amount of time. v

Example: a simple algorithm Algorithm to find the largest of three numbers a, b, c: Assignment operator s : = k means “copy the value of k into s” 1. x: = a p 2. If b > x then x: = b p 3. If c > x then x: = c A trace is a check of the algorithm for specific values of a, b and c p

Notation for algorithms PSEUDOCODE: Instructions given in a generic language similar to a computer language such as C++ or Pascal. (no fuss over syntax) n Else n Procedure n Return n If-then, action n While loop n If-then-else n For loop n begin n End

Problem Solving p Searching p Sorting

Tracing Output A=1 B=1 Repeat until B 10 B=2 A-2 A=A+3

Example Function F(X) If X > 0 then R=X 2 + 1 Else If X < 3 R=2 X+6 Else R=X+7 Return (R) F(3)?

Example A=1 B=0 While (A 10) X=X+1 A=A+1

Example (Sorting) Let assume there are two integer numbers: 3, 1 Aim: Sort these numbers based on ascending order Location[1] =3, Location[2]=1 If Location[1] is greater than Location[2] then Begin temp=Location[1]=Location[2]=temp End

Example (Searching) p Linear Search Begin i=1 While ((i <= n) and (x <> A[i])) i=i+1 if i <= n then location = i else location = - 1; (where x is the item we are searching for; A is the array of where the item should be searched and location is a variable that will hold the position of where the item is, if it is found and -1 is the item is not found)

Recursive algorithms q A recursive procedure is a procedure that invokes itself q Example: given a positive integer n, factorial of n is defined as the product of n by all numbers less than n and greater than 0. Notation: n! = n(n-1)(n-2)… 3. 2. 1 Observe that n! = n(n-1)(n-2)!, etc. q A recursive algorithm is an algorithm that contains a recursive procedure q

factorial(n: positive integer) if n=1 then factorial(n)=1 else factorial(n)=n*factorial(n-1)

Fibonacci sequence Leonardo Fibonacci (Pisa, Italy, ca. 1170 -1250) q Fibonacci sequence f 1, f 2, … defined recursively as follows: f 1 = 1 f 2 = 2 fn = fn-1 + fn-2 for n > 3 q First terms of the sequence are: 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, … q

Complexity of algorithms Analysis : refers to the process of deriving estimates for the time and space needed to execute the algorithm. p Complexity: the amount of time and/or space needed to execute the algorithm. p Complexity depends on many factors: data representation type, kind of computer, computer language used, etc. p

Types of complexity Time needed to execute an algorithm is a function of the input. n It is difficult to obtain an explicit formula for this function but we can try to determine : n p Best-case time = minimum time needed to execute the algorithm for inputs of size n p Worst-case time = maximum time needed to execute the algorithm for inputs of size n p Average-case time = average time needed

Big-O Notation Big-O has been used in mathematics for century, and in CS it is used in the analysis of algorithms p Popularized in CS by Donald Knuth. p

Notational Issues Big-O notation is a way of comparing functions. Notation unconventional: EG: 3 x 3 + 5 x 2 – 9 = O (x 3) Doesn’t mean “ 3 x 3 + 5 x 2 – 9 equals the function O (x 3)” Which actually means “ 3 x 3+5 x 2 – 9 is dominated by x 3” Read as: “ 3 x 3+5 x 2 – 9 is big-Oh of x 3”

Intuitive Notion of Big-O Asymptotic notation captures behavior of functions for large values of x. EG: Dominant term of 3 x 3+5 x 2 – 9 is x 3. As x becomes larger and larger, other terms become insignificant and only x 3 remains in the picture:

Intuitive Notion of Big-O domain – [0, 2] y = 3 x 3+5 x 2 – 9 y=x 3 y=x 2 y=x

Intuitive Notion of Big-O domain – [0, 5] y = 3 x 3+5 x 2 – 9 y=x 3 y=x 2 y=x

Intuitive Notion of Big-O domain – [0, 10] y = 3 x 3+5 x 2 – 9 y=x 3 y=x 2 y=x

Big-O. Formal Definition f (x ) is asymptotically dominated by g (x ) if there’s a constant multiple of g (x ) bigger than f (x ) as x goes to infinity: DEF: Let f , g be functions with the set of R 0 or N to set of R. If there are constants C and k such x > k, |f (x )| C |g (x )| then we write: f (x ) = O ( g (x ) ) p (Big- O is actually an estimate of average running time of an algorithm. )

Intuitive Notion of Big-O domain – [0, 100] y = 3 x 3+5 x 2 – 9 y=x 3 y=x 2 y=x

Common Terminology of Complexity O(1) O(log n) O(nb) O(bn), where b > 1 O(n!) Terminology Constant Logarithmic Linear n log n Polynomial Exponential Factorial

Example: To find Big. O of Linear search algo p Describe the average-case performance of linear search algo, assuming that the element x is in the list. n Solution : There are n types of possible inputs when x is known to be in the list. If x is the first term of the list, 3 comparison is needed (one to determine whether the end of the list has been reached, one to compare x and the first term and one outside the loop). If x is the second term then 2 more comparisons is needed, which total up to 5 comparisons and so on, where if x is at the ith term a total of 2 i + 1 comparisons are needed.

p Hence the average number of camparisons used : 3+5+7+…+ (2 n + 1) = 2(1+2+3+. . +n)+ n n n 1+2+3+…+ n = n(n+1) 2 therefore : 2[n(n+1) /2] + 1 = n + 2 n which is O(n).