Module I Introduction Algorithm Sequence of unambiguous instructions

Module - I

Introduction • Algorithm • Sequence of unambiguous instructions for solving a problem. • Notion of algorithms Problem Algorithm Input Computer Output

Algorithm Specification • Input : Zero or More • Output : At least one quantity • Definiteness : Clear and Producible • Finiteness : Termination after a finite number of steps • Effectiveness : Every instruction must be very basic and must be feasible.

Analysis Framework • Time efficiency : Algorithms run time • Space efficiency : Extra space requirement by algorithms.

Measuring Running Time • Running time : Execution of Basic operation input size T(n) ≈ cop. C(n) running time execution time for basic operation Number of times basic operation is executed

Best-case, average-case, worst-case If execution of an algorithms basic operation varies based on input elements: • Worst case: Cworst(n) – Maximum execution over n inputs • Best case: Cbest(n) – Minimum execution over n inputs • Average case: Cavg(n) – Average execution over n inputs

Example: Sequential search • Worst case : (n) • Best case: 1 • Average case: (n+1)/2

Understand the problem Decide on : computational means, Exact vs approximate solution Data structures Algorithm design technique Design an algorithm Prove correctness Analyze the algorithm Code the algorithm

What does it mean to understand the problem? • What are the problem objects? • What are the operations applied to the objects? Deciding on computational means • How the objects would be represented? • How the operations would be implemented? 8

Design an algorithm • Build a computational model of the solving process • Prove correctness • Correct output for every legitimate input in finite time • Based on correct math formula • By induction 9

Analysis of the algorithm Efficiency: Time and space Simplicity Generality: Range of inputs, special cases Optimality: No other algorithm can do better 10

Fundamental data structures Linear data structures • Array • Linked list • Stack • Queue Operations: search, delete, insert Implementation: static, dynamic 11

Fundamental data structures Non-linear data structures • Graphs • Trees : connected graph without cycles • Rooted trees • Ordered trees • Binary trees Graph representation: adjacency lists, adjacency matrix Tree representation: as graphs; binary nodes 12

Fundamental data structures Sets, Bags, Dictionaries • Set: unordered collection of distinct elements Operations: membership, union, intersection Representation: bit string; linear structure • Bag: unordered collection, elements may be repeated • Dictionary: a bag with operations search, add, delete 13

Values of some important functions as n

Asymptotic Notations Big-oh Definition: t(n) is in O(g(n)) if order of growth of t(n) ≤ order of growth of g(n) (within constant multiple), i. e. , there exist positive constant c and non-negative integer n 0 such that t(n) ≤ c g(n) for every n ≥ n 0

• Big-omega Definition: t(n) is in O(g(n)) if order of growth of t(n) ≥ order of growth of g(n) (within constant multiple), i. e. , there exist positive constant c and non-negative integer n 0 such that t(n) ≥ c g(n) for every n ≥ n 0

• Big-theta Definition: t(n) is in O(g(n)) if order of growth of t(n) both ≤ & ≥ order of growth of g(n) (within constant multiple), i. e. , there exist positive constant c and non-negative integer n 0 such that c 1 g(n) ≤t(n) ≤c 2 g(n) for every n ≥ n 0

Some properties of asymptotic order of growth • f(n) O(f(n)) • f(n) O(g(n)) iff g(n) (f(n)) • If f (n) O(g (n)) and g(n) O(h(n)) , then f(n) O(h(n)) Note similarity with a ≤ b • If f 1(n) O(g 1(n)) and f 2(n) O(g 2(n)) , then f 1(n) + f 2(n) O(max{g 1(n), g 2(n)})

Basic asymptotic efficiency classes 1 constant log n logarithmic n linear n log n n-log-n n 2 quadratic n 3 cubic 2 n exponential n! factorial

Mathematical Analysis of nonrecursive algorithms • Deciding the input parameter size • Identifying algorithm’s basic operation • Finding the reason if the basic operation is to be executed more than once • Setting up a sum or recurrence relation for the number of times the basic operation is executed • Finding the complexity function and order of growth using the standard formulae.

Useful summation formulas and rules l i u 1 = 1+1+…+1 = u - l + 1 In particular, l i u 1 = n - 1 + 1 = n (n) 1 i n i = 1+2+…+n = n(n+1)/2 n 2/2 (n 2) 1 i n i 2 = 12+22+…+n 2 = n(n+1)(2 n+1)/6 n 3/3 (n 3) 0 i n ai = 1 + a +…+ an = (an+1 - 1)/(a - 1) for any a 1 In particular, 0 i n 2 i = 20 + 21 +…+ 2 n = 2 n+1 - 1 (2 n ) (ai ± bi ) = ai ± bi m+1 i uai cai = c ai l i uai = l i mai +

Example 1: Maximum element

Mathematical analysis of nonrecursive algorithms •

Example 2: Element uniqueness problem

Mathematical analysis of nonrecursive algorithms •

Mathematical analysis of nonrecursive algorithms •

Example 3: Matrix multiplication

Mathematical analysis of nonrecursive algorithms •

Example 5: Counting binary digits

Mathematical Analysis of Recursive Algorithms • Deciding the input parameters size. • Identifying the algorithm’s basic operation. • Finding the reason if the basic operation is to be executed more than once • Set up a recurrence relation with an appropriate initial condition expressing the number of times the basic op. is executed. • Solve the recurrence (or, at the very least, establish its solution’s order of growth) by backward substitutions or another method.

Example 1: Recursive evaluation of n! Definition: n ! = 1 2 … (n-1) n for n ≥ 1 and 0! = 1 Recursive definition of n!: F(n) = F(n-1) n for n ≥ 1 and F(0) = 1 Size: Basic operation: Recurrence relation: T(n) = T(n-1) + 1, T(0) = 0

The Tower of Hanoi Puzzle Algorithm : TOH(n, S, T, D) // Input : Number od discs //Output : The sequence of movements if n>0 TOH( n-1, S, D, T) move one disc from S to D TOH(n-1, T, S, D) Recurrence relation : T(n) = TM(n-1) + 1, T(1) = 1

Example 3: Counting #bits

Fibonacci numbers The Fibonacci numbers: 0, 1, 1, 2, 3, 5, 8, 13, 21, … The Fibonacci recurrence: F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1 General 2 nd order linear homogeneous recurrence with constant coefficients: a. X(n) + b. X(n-1) + c. X(n-2) = 0

Algorithm : Fib(n) // Finding nth element of Fibonacci series //i/p : A non-negative integer ‘n’ //o/p : nth element of the Fibonacci series if n=0 || n=1 return n; Else return (fib(n-1) + fib(n-2))

Application to the Fibonacci numbers T(n) = T(n-1) + T(n-2) or T(n) - T(n-1) - T(n-2) = 0 Particular solution for T(0) =0, T(1)=1 General 2 nd order linear homogeneous recurrence with constant coefficients: a. X(n) + b. X(n-1) + c. X(n-2) = 0 • Set up the characteristic equation (quadratic) ar 2 + br + c = 0 a=1, b=-1 and c=-1 r 2 -r -1 = 0

• 0 0 1 1 n n

Important Problem Types • Sorting • Searching • String processing • Graph problems Representation of Graphs G< E, N> • Linked lists • Adjacency matrix • Combinatorial problems • Geometric problems • Numerical problems

Fundamental Data Structures • Stack • Queue • Graphs • Trees • Sets & Dictionaries