CS 440 Theory of Algorithms CS 468 Algorithms

CS 440 Theory of Algorithms / CS 468 Algorithms in Bioinformatics Introduction Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

● “The most valuable acquisitions in a scientific or technical education are the general-purpose mental tools which remain serviceable for a lifetime. ” – George Forsythe, What to Do Till the Computer Scientist Comes, 1967 ● "A person well trained in computer science knows how to deal with algorithms; how to construct them, manipulate them, understand them, analyze them. This knowledge prepares him for much more than writing good computer programs; it is a general-purpose mental tool which will be a definite aid to his understanding of other subjects, whether they be chemistry, linguistics, or music, etc. " – Donald Knuth, Selected Papers on Computer Science, 1996 ● “Two ideas lie gleaming on the jeweler's velvet. The first is the calculus, the second, the algorithm. The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible; but it has been the algorithm that has made possible the modern world. ” – David Berlinski, The Advent of the Algorithm, 2000 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -1

What is an Algorithm? ● An algorithm is a sequence of unambiguous instructions for solving a problem, i. e. , for obtaining a required output for any legitimate input in a finite amount of time. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -2

Are algorithms just of theoretical intesrets? ● Application areas of algorithms Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -3

What is an algorithm? ● 1. Recipe, process, method, technique, procedure, routine, … with following requirements: Finiteness • 2. Definiteness • 3. valid inputs are clearly specified Output • 5. rigorously and unambiguously specified Input • 4. terminates after a finite number of steps can be proved to produce the correct output given a valid input Effectiveness • steps are sufficiently simple and basic Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -4

Why study algorithms? ● Theoretical importance • The core of computer science ● Practical importance • A practitioner’s toolkit of known algorithms • Framework for designing and analyzing algorithms for new problems Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -5

Historical Perspective ● Euclid’s algorithm for finding the greatest common divisor ● Muhammad ibn Musa al-Khwarizmi – 9 th century mathematician Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -6

Notion of Algorithm problem algorithm input “computer” output Algorithmic solution Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -7

Example of Computational Problem: Sorting ● Statement of problem: • Input: A sequence of n numbers <a 1, a 2, …, an> • Output: A reordering of the input sequence <a´ 1, a´ 2, …, a´n> so that a´i ≤ a´j whenever i < j ● Instance: The sequence <5, 3, 2, 8, 3> ● Algorithms: • • Selection sort Insertion sort Merge sort (many others) Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -8

Selection Sort ● Input: array a[1], …, a[n] ● Output: array a sorted in non-decreasing order ● Algorithm: for i=1 to n swap a[i] with smallest of a[i], …a[n] • see also pseudocode, section 3. 1 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -9

Euclid’s Algorithm Problem: Find gcd(m, n), the greatest common divisor of two nonnegative, not both zero integers m and n Examples: gcd(60, 24) = 12, gcd(60, 0) = 60, gcd(0, 0) = ? Euclid’s algorithm is based on repeated application of equality gcd(m, n) = gcd(n, m mod n) until the second number becomes 0, which makes the problem trivial. Example: gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. 1 -10

Two descriptions of Euclid’s algorithm Step 1 If n = 0, return m and stop; otherwise go to Step 2 Divide m by n and assign the value fo the remainder to r Step 3 Assign the value of n to m and the value of r to n. Go to Step 1. while n ≠ 0 do r ← m mod n m← n n←r return m Copyright © 2007 Pearson Addison-Wesley. All rights reserved. 1 -11

Other methods for computing gcd(m, n) Consecutive integer checking algorithm Step 1 Assign the value of min{m, n} to t Step 2 Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4 Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4 Decrease t by 1 and go to Step 2 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. 1 -12

Other methods for gcd(m, n) [cont. ] Middle-school procedure Step 1 Step 2 Step 3 Step 4 Find the prime factorization of m Find the prime factorization of n Find all the common prime factors Compute the product of all the common prime factors and return it as gcd(m, n) Is this an algorithm? Copyright © 2007 Pearson Addison-Wesley. All rights reserved. 1 -13

Sieve of Eratosthenes Input: Integer n ≥ 2 Output: List of primes less than or equal to n for p ← 2 to n do A[p] ← p for p ← 2 to n do if A[p] 0 //p hasn’t been previously eliminated from the list j ← p* p while j ≤ n do A[j] ← 0 //mark element as eliminated j←j+p Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. 1 -14 http: //www. cs. princeton. edu/introcs/14 array/Prime. Sieve. java. html

More Problems ● Multiplication of two integers (e. g. , 1526 x 77) • American way • British way • “Chinese” way (http: //www. youtube. com/results? search_query=multiplication&sea rch_type=&aq=6&oq=multi ) • Egyptian way ● Coin change problem • Greedy • Brute force (exhaustive search) • Other techniques? Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -15

Problem Solving Problem 1: Old world puzzle A peasant finds himself on a riverbank with a wolf, a goat, and a head of cabbage. He needs to transport all three to the other side of the river in his boat. However, the boat has room for only the peasant himself and one other item. In his absence, the wolf would eat the goat, and the goat would eat the cabbage. Solve this problem, or prove it has no solution. * The peasant is vegetarian who doesn’t like to eat cabbage. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. 1 -16

Problem Solving Problem 2: New world puzzle There are four people, A, B, C, and D, who want to cross a bridge; they all begin on the same side. You have 17 minutes to get them all across to the other side. It is night, and they have one flashlight. A maximum of two people can cross the bridge at one time. Any party that crosses, either one or two people, must have the flashlight with them. The flashlight must be walked back and forth; it cannot be thrown, for example. A takes 1 minute to cross the bridge, B takes 2 minutes, C takes 5 minutes, and D takes 10 minutes. A pair must walk together at the rate of the slower person’s pace. For example, if A and D walk across first, 10 minutes have elapsed when they get to the other side of the bridge. If D returns the flashlight, a total of 20 minutes have passed and you have failed the mission. Solve this problem, or prove it has no solution. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. 1 -17

Fundamentals of Algorithmic Problem Solving Copyright © 2007 Pearson Addison-Wesley. All rights reserved. 1 - 18

Some Well-known Computational Problems Problem Categories Problems ● Graph Problem ● Sorting ● String Processing ● Searching ● Combinatorial Problems ● Shortest paths in a graph ● Geometric Problems ● Minimum spanning tree ● Numerical Problems ● Primality testing ● Traveling salesman problem ● Knapsack problem ● Chess ● Towers of Hanoi ● Program termination Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -19

Basic Issues Related to Algorithms ● How to design algorithms ● How to express algorithms ● Proving correctness ● Efficiency • Theoretical analysis • Empirical analysis ● Optimality Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -20

Algorithm design strategies ● Brute force ● Greedy approach ● Divide and conquer ● Dynamic programming ● Decrease and conquer ● Backtracking and Branch and bound ● Transform and conquer ● Space and time tradeoffs There are more, for example, Machine learning, Randomized algorithms, Genetic algorithms, etc. Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -21

Analysis of Algorithms ● How good is the algorithm? • Correctness • Time efficiency • Space efficiency ● Does there exist a better algorithm? • Lower bounds • Optimality Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Design and Analysis of Algorithms - Chapter 1 1 -22