Algorithm An algorithm is a sequence of unambiguous

Algorithm • An algorithm is a sequence of unambiguous instructions for solving a problem, i. e. , for obtaining a required output for any legitimate input in a finite amount of time. UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Features of Algorithm • “Besides merely being a finite set of rules that gives a sequence of operations for solving a specific type of problem, an algorithm has the five important features” [Knuth 1] • finiteness (otherwise: computational method) » termination • definiteness » precise definition of each step • input (zero or more) • output (one or more) • effectiveness » “Its operations must all be sufficiently basic that they can in principle be done exactly and in a finite length of time by someone using pencil and paper” [Knuth 1] UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Historical Perspective • Muhammad ibn Musa al-Khwarizmi – 9 th century mathematician www. lib. virginia. edu/science/parshall/khwariz. html • Euclid’s algorithm for finding the greatest common divisor UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Greatest Common Divisor (GCD) Algorithm 1 • Step 1 Assign the value of min{m, n} to t • Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, to Step 4 • Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4 • Step 4 Decrease the value of t by 1. Go to Step 2 • Note: m and n are positive integers UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Correctness • The algorithm terminates, because t is decreased by 1 each time we go through step 4, 1 divides any integer, and t eventually will reach 1 unless the algorithm stops earlier. • The algorithm is partially correct, because when it returns t as the answer, t is the minimum value that divides both m and n UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

GCD Procedure 2 • Step 1 Find the prime factors of m • Step 2 Find the prime factors of n • Step 3 Identify all the common factors in the two prime expansions found in Steps 1 and 2. If p is a common factor repeated i times in m and j times in n, assign to p the multiplicity min{i, j} • Step 4 Compute the product of all the common factors with their multiplicities and return it as the GCD of m and n • Note: as written, the algorithm requires than m and n be integers greater than 1, since 1 is not a prime UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Procedure 2 or Algorithm 2? • Procedure 2 is not an algorithm unless we can provide an effective way to find prime factors of a number • The sieve of Eratosthenes is an algorithm that provides such an effective procedure UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Euclid’s Algorithm • E 0 [Ensure m geq n] If m lt n, exchange m with n • E 1 [Find Remainder] Divide m by n and let r be the remainder (We will have 0 leq r lt n) • E 2 [Is it zero? ] If r=0, the algorithm terminates; n is the answer • E 3 [Reduce] Set m to n, n to r, and go back to E 1 UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Termination of Euclid’s Algorithm • The second number of the pair gets smaller with each iteration and cannot become negative: indeed, the new value of n is r = m mod n, which is always smaller than n. Eventually, r becomes zero, and the algorithms stops. UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Correctness of Euclid’s Algorithm • After step E 1, we have m = qn + r, for some integer q. If r = 0, then m is a multiple of n, and clearly in such a case n is the GCD of m and n. If r !=0, note that any number that divides both m and n must divide m – qn = r, and any number that divides both n and r must divide qn + r = m; so the set of divisors of {m, n} is the same as the set of divisors of {n, r}. In particular, the GCD of {m, n} is the same as the GCD of {n, r}. Therefore, E 3 does not change the answer to the original problem. [Knuth 1] UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Algorithm Design Techniques • Euclid’s algorithm is an example of a “Decrease and Conquer” algorithm: it works by replacing its instance by a simpler one (in step E 3) • It can be shown (exercise 5. 6 b) that the instance size, when measured by the second parameter, n, is always reduced by at least a factor of 2 after two successive iterations of Euclid’s algorithm • In Euclid’s original, the remainder m mod n is computed by repeated subtraction of n from m • Step E 0 is not necessary for correctness; it improves time efficiency UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Notion of algorithm problem algorithm input “computer” output Algorithmic solution UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 12 Department of Computer Science and Engineering

Example of computational problem: sorting • Statement of problem: – Input: A sequence of n numbers <a 1, a 2, …, an> – Output: A reordering of the input sequence <a´ 1, a´ 2, …, a´n> so that a´i ≤ a´j whenever i < j • Instance: The sequence <5, 3, 2, 8, 3> • Algorithms: – Selection sort – Insertion sort – Merge sort – (many others) UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Selection Sort • Input: array a[1], …, a[n] • Output: array a sorted in non-decreasing order • Algorithm: • for i=1 to n swap a[i] with smallest of a[i+1], …a[n] • see also pseudocode, section 3. 1 UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Sorting: Some Terms • Key • Stable sorting • In place sorting UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Graphs: Some Terms • • • Graph Vertices, edges Undirected, directed Loops Complete, dense, sparse Paths: sequences of vertices or of edges? Path length: number of edges in path Walk: repeated vertices and edges allowed Path: repeated vertices are allowed, repeated edges are not allowed • Simple path: neither repeated vertices nor repeated edges are allowed • Cycle: simple path that starts and ends at the same vertex UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Some Well-known Computational Problems • • • Sorting Searching Shortest paths in a graph Minimum spanning tree Primality testing Traveling salesman problem Knapsack problem Chess Towers of Hanoi Program termination UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Basic Issues Related to Algorithms • How to design algorithms • How to express algorithms • Proving correctness • Efficiency – Theoretical analysis – Empirical analysis • Optimality UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Algorithm design strategies See http: //www. aw-bc. com/info/levitin/taxonomy. html This is not a unique taxonomy. • Brute force • Greedy approach • Divide and conquer • Dynamic programming • Decrease and conquer • Backtracking and Branch and bound • Transform and conquer UNIVERSITY OF SOUTH CAROLINA • Space and time tradeoffs Design and Analysis of Algorithms - Chapter 1 19 Department of Computer Science and Engineering

Analysis of Algorithms • How good is the algorithm? – Correctness – Time efficiency – Space efficiency • Does there exist a better algorithm? – Lower bounds – Optimality UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

What is an algorithm? • Recipe, process, method, technique, procedure, routine, … with the following requirements: 1. Finiteness b terminates after a finite number of steps 2. Definiteness b rigorously and unambiguously specified 3. Input b valid inputs are clearly specified 4. Output b can be proved to produce the correct output given a valid input 5. Effectiveness b steps are sufficiently simple and basic UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Why study algorithms? • Theoretical importance – the core of computer science • Practical importance – A practitioner’s toolkit of known algorithms – Framework for designing and analyzing algorithms for new problems UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Correctness • Termination • Well-founded sets: find a quantity that is never negative and that always decreases as the algorithm is executed • Partial Correctness • For recursive algorithms: induction • For iterative algorithms: axiomatic semantics, loop invariants UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering

Complexity • Space complexity • Time complexity • For iterative algorithms: sums • For recursive algorithms: recurrence relations UNIVERSITY OF SOUTH CAROLINA Design and Analysis of Algorithms - Chapter 1 Department of Computer Science and Engineering