CS 415 Algorithm Analysis Lecture T Th 2

CS 415 Algorithm Analysis Lecture T Th 2 – 3: 50 PM, Salazar Hall 2023 Instructor: Bala Ravikumar Office: 116 I Darwin Hall Phone: 664 3335 E-mail through piazza Course Web site: https: //piazza. com/sonoma/fall 2014/cs 415/resources

Text book: Published in 2006 Authors: Dasgupta, Papadimitrou and Vazirani (UCSD, UCB) Other references: • Algorithm Design by Kleinberg and Tardos (used as text last time when I taught – Spring 2011) • Cormen et al. Introduction to Algorithms (Third edition) • Algorithms A creative approach by Udi Manber And numerous other sources. 2

Lecture videos • Leiserson and Demaine, Devdas Course taught at MIT • Tim Roughgarden, coursera course on Algorithms • Sedgewick and Wayne, coursera course on Algorithms • Papadimitriou (author of text) Comp Science 170 Berkeley and many others. Warning: I will assign videos to watch that either covers (roughly) the topic of an upcoming lecture or the background needed for an upcoming lecture. Please watch before coming to class. 3

Course policies, grading etc. Short quizzes and class participation – 10 points (There will be a quiz about once a week. The quiz will be a direct application of the topic being covered. ) Home work assignments - 10 points. Most home work problems will be from the text. They are intended to deepen your understanding of the material presented in class and will play a key role in prepare you for the mid-term and final exams. You can discuss the home work problems with others, but you should write up the solution on your own. Two mid-semester tests - 30 points (The mid-semester tests will be openbook/notes, and will be in class. ) Projects - 25 points. There will be some implementation projects – including a final project for which each (group) will choose a problem from an assigned list. Final Examination - 25 points. This will be comprehensive and will take place at the pre-specified time slot. You can find the date from the university’s calendar. 4

What is an algorithm? • computer science is often described as a study of algorithms. • Algorithm is software – without the rigid syntax rules. Intended for human reading, but not machines. • What is the exact definition? One way is to create an extremely simple language with only one or two instructions. Another one is in terms of something called Turing machine. • Algorithms are much older than computers. How old are they? 5

An algorithm – more than 3500 years old Rhind Mathematical Papyrus British Museum, London One of the algorithms presented is for multiplying two numbers: 59 41 _________ 1 41 x 2 82 x 4 164 8 328 x 16 656 x 32 1312 x _________ 2419 6

An algorithm from Al-Khwarizmi’s book (about 1200 years old) 7

Historically famous algorithms Classical (~ 2000 years) • Finding area of triangles and other shapes (Heron’s formula) • Finding GCD of integers (Euclid’s algorithm) • Solving linear system of equations, quadratic equation etc. Intermediate (~ 500 years old) • logarithm tables • Roots of polynomials, solution of differential equations etc. • calculus Applications? • Dividing land (after floods) • Calculating trajectories of planets • architecture (buildings, dams etc. ) • navigation (finding direction in deep sea) • town planning, governance (tax rate, interest etc. ) 8

Famous modern algorithms (20 th century algorithms) • quick-sort There are probably many other candidates. Can you name one ? • Dijkstra’s algorithm • RSA (for encryption) • Fast Fourier Transform • Simplex algorithm • Huffman coding algorithm, LZW algorithm (both for compression) • Hamming code (corrects errors) 9

Nine algorithms that changed the future 2. Search Engine Indexing: Finding Needles in the World's Biggest Haystack 3. Page. Rank: The Technology That Launched Google 4. Public Key Cryptography: Sending Secrets on a Postcard 5. Error-Correcting Codes: Mistakes That Fix Themselves 6. Pattern Recognition: Learning from Experience 7. Data Compression: Something for Nothing 8. Databases: The Quest for Consistency 9. Digital Signatures: Who Really Wrote This Software? 10. What Is Computable? Princeton University Press Watch the video: http: //press. princeton. edu/titles/9528. html 10

Algorithm for integer addition • Earlier we looked at a 2500 old algorithm for multiplying two integers – that assumes that you know how to add two integers, divide an integer by 2 and know to compare two integers. • What is the algorithm for integer addition? Consider binary addition: • Given two binary numbers x and y, how long does our algorithm take to add them? We want the answer expressed as a function of the number of input bits. • The basic operation for this operation are bit-level operations: adding two 1 bit numbers is considered as a single operation. 11

Algorithm for multiplication – school method • What is the number of bit level operations as a function of input? Assume that the input numbers are n bits long. • Cost to generate all the rows + cost to add them up. • What about the number of storage bits needed to implement the above algorithm? 12

Complexity of the algorithm Time complexity is one of the central issues that we will be discussing. For every algorithm, we want to measure its time complexity. Rough definition of time complexity: The number of operations performed by the algorithm. What operations? • we could count every operation, e. g. assignment, comparison, addition etc. • but what is an operation? E. g. % or / may require many CPU cycles, or machine instructions. • one approach is to fix an instruction set, and architecture and count the number of machine instructions. • another approach is to focus on the most expensive operation. 13

Multiplication algorithm 2 – the Egyptian Algorithm We will write this algorithm using pseudo-code: Input: two positive integers A and B. Output: A * B Operations allowed: (a) Addition of two integers (b) comparison between two integers. (cost O(n) bit-level operations) and (c) check if an integer is odd or even (costs O(1) bit-level operation) • Recall how the algorithm works. • Example: 14

Multiplication algorithm 2 – the Egyptian Algorithm We will write this algorithm using pseudo-code: Input: two positive integers A and B. Output: A * B Operations allowed: (a) Addition of two integers (b) comparison between two integers (c) divide a number by 2 (integer division) (each operation costs O(n) bit-level operation) and (c) check if an integer is odd or even (costs O(1) bit-level operation) Input: A, B 1. prod = 0 2. while (B > 0) if (B is odd) prod = prod + A A=A+A B=B/2 end while 3. output prod 15

Multiplication algorithm 2 – the Egyptian Algorithm Input: A, B 1. prod = 0 2. while (B > 0) if (B is odd) prod = prod + A A=A+A B=B/2 end while 3. output prod What is the time complexity of this algorithm? Cost model: assume the inputs A, B are in binary with n bits each. • Addition two n-bit numbers take O(n) operations • Testing if a number B > 0 take O(n) operations. • B = B/2 takes 1 operation. Testing if B is odd takes 1 operation. 16

Euclid’s algorithm for GCD computation Euclid’s algorithm for finding the GCD of two positive integers. What is the definition of GCD? What is the main application of GCD? (modular division) First recall the following concepts: • Divisor: x is said to be a divisor of y if the remainder is 0 when y is divided by x. • Common divisor of y and z: x is a common divisor of y and z if it is a divisor of both y and z. • GCD: greatest common divisor of two numbers y and z. 17

Euclid’s algorithm for GCD computation To define GCD, we first need the definition of divisor, then common divisor and finally greatest common divisor. p is a divisor of q if p is a multiple of q, or the remainder when q is divided by p is 0. (We write p | q) Example: 4 | 12, 15 | 30, but 12 | 30. p is a common divisor of x and y if p|x and p|y. p is the GCD of x and y if p is the largest among the common divisors of x and y. 18

Euclid’s algorithm for GCD computation Algorithm GCD input: x, y – positive integers output: m – GCD of x and y Step 1: if y > x, swap x and y. Step 2: if y = 0, set m = x and stop. Step 3: r = remainder when x divided by y x=y y=r Step 4: go to step 2 Exercise: compute the GCD of 948 and 156. 19

Correctness of the algorithm • Does the algorithm stop on all inputs? (i. e. , can we show that there is no infinite loop? ) if the algorithm has no loops, this is not a problem. But most algorithms do. (Euclid’s does!) • Does it output the correct result for all inputs? Work out some small test cases and see if we get the correct result. Even this is tricky. We need to know a different way (than using the present algorithm) to get the answer so that we can cross-check. But proving correctness requires mathematical argument, not just some test cases. 20

Recursive version of the algorithm 21

Proving the correctness of Euclid’s algorithm Two steps involved in proving that an algorithm is correct. • Show that Euclid’s algorithm always terminates. Will be done in class. • Show that Euclid’s algorithm always produces the correct output. Will be done in class. 22

Complexity of the algorithm Many ways to measure complexity: - best-case: of all the inputs of size N, the input that makes the algorithm perform the fewest number of divisions. What is the best case for Euclid’s algorithm? - worst-case: The one of size N for which the algorithm performs the maximum number of operations. - average-case: add up the complexity for all inputs of size N, and divide it by the number of such inputs. The most common one used in this course is the worstcase complexity. 23

Complexity of Euclid’s algorithm If the inputs A and B are N bits, how many recursive calls will it take (in the worst-case) before the base case is reached? What about the best-case? This is easy to answer. Claim: In the worst-case, it will take at most 2 n recursive calls. Proof will be presented in class. Complexity analysis: Each call involves a mod operation followed by another recursive call. Cost of mod operation is O(n 2) if the inputs are n bits long. Total complexity is O(n 3). 24

Algorithm design techniques At least two approaches: • problem-oriented • shortest-path • sorting • linear programming etc. • technique-oriented • greedy method • dynamic programming • divide and conquer • transformation etc. 25

Reduction – mapping one problem to another Effectiveness of problem-oriented method is • if you know how to solve problem A, then you know how to solve problem B. • most problems can be solved by breaking into sub problems each of which is one of the standard problems for which well-crafted solution exists. • a powerful way to reduce one problem to another is “problem transformation”. 26

Transformation • consider a two player game in which players alternately choose a number between 1 and 9. (Each number can be chosen at most once. ) At any stage, if a player has three numbers that add to 15, he/she has won. • What will be your strategy? 27