CS 146 Data Structures and Algorithms June 2

CS 146: Data Structures and Algorithms June 2 Class Meeting Department of Computer Science San Jose State University Summer 2015 Instructor: Ron Mak www. cs. sjsu. edu/~mak

Basic Info o Office hours n n o Tu. Th 2: 00 – 3: 00 PM MH 413 Class website n n http: //www. cs. sjsu. edu/~mak/ Green sheet Assignments Lecture notes Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 2

Course Objectives o Ensure that students are familiar with ways to implement elementary data structures and their associated algorithms. o Introduce students to the implementation of more complex data structures and their associated algorithms. o Acquaint students with advanced sorting techniques (radix sort, heap sort, merge sort, quicksort). Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 3

Course Objectives, cont’d o Teach students how to determine the time complexity of algorithms. o Introduce students to algorithm design techniques. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 4

Student Learning Outcomes o o o Implement lists, stacks, queues, search trees, heaps, union-find ADT, and graphs and use these data structures in programs they design. Prove basic properties of trees and graphs. Perform breadth-first search and depth-first search on directed as well as undirected graphs. Use advanced sorting techniques (radix sort, heap sort, merge sort, quicksort). Determine the running time of an algorithm in terms of asymptotic notation. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 5

Student Learning Outcomes o o o Solve recurrence relations representing the running time of an algorithm designed using a divide-and-conquer strategy Comprehend algorithms designed using greedy, divide-and-conquer, and dynamic programming techniques Comprehend the basic concept of NPcompleteness and realize that they may not be able to efficiently solve all problems they encounter in their careers Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 6

Take Roll! Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 7

Introduction to Algorithm Analysis o To analyze an algorithm, we are measuring it. o A convenient measure must be: n n o Measuring a resource we care about (elapsed time, memory usage, etc. ). Quantitative, to make comparisons possible. Easy to compute. A good predictor of the “goodness” of the algorithm. Mostly in this class, we will be concerned with elapsed time. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 8

Example: Reading Books o o Algorithm: Read a book. Measure: Length of time to read a book. o Given a set of books to read, can we predict how long it will take to read each one, without actually reading it? o Possible ways to compute reading time: n n weight of the book physical size (width, height, thickness) of the book total number of words total number of pages Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 9

Introduction to Algorithm Analysis, cont’d o Our concern generally is not how long a particular run of an algorithm will take, but how well the algorithm scales. o How does the run time increase as the amount of input increases. n Example: How does the reading time of a book increase as the number of pages increases? n Example: How does the run time of a particular sort algorithm increase as the number of items to be sorted increases? Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 10

Introduction to Algorithm Analysis, cont’d o When we compare two algorithms, we want to compare how well they scale. o Can we do this comparison without actually running the algorithms? Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 11

How Well Does an Algorithm Scale? Data Structures and Algorithms in Java, 3 rd ed. by Mark Allen Weiss Pearson Education, Inc. , 2012 ISBN 978 -0 -13 -257627 -7 Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 12

How Well Does an Algorithm Scale? cont’d Data Structures and Algorithms in Java, 3 rd ed. by Mark Allen Weiss Pearson Education, Inc. , 2012 ISBN 978 -0 -13 -257627 -7 Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 13

How Well Does an Algorithm Scale? cont’d Data Structures and Algorithms in Java, 3 rd ed. by Mark Allen Weiss Pearson Education, Inc. , 2012 ISBN 978 -0 -13 -257627 -7 Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 14

How Well Does an Algorithm Scale? cont’d Data Structures and Algorithms in Java, 3 rd ed. by Mark Allen Weiss Pearson Education, Inc. , 2012 ISBN 978 -0 -13 -257627 -7 Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 15

Towers of Hanoi o Goal: Move the stack of disks from the source pin to the destination pin. n n n You can move only one disk at a time. You cannot put a larger disk on top of a smaller disk. Use third pin for temporary disk storage. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 16

Towers of Hanoi: Solve Recursively! o Label the pins A, B, and C. n n n o Solve one disk (source destination) n o A: source B: temporary C: destination Move disk from A to C (source destination) Solve two disks (source destination) n n n Move disk from A to B (source temp) Move disk from A to C (source destination) Move disk from B to C (temp destination) Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 17

Towers of Hanoi: Solve Recursively! cont’d o Solve three disks (source destination) n n n Solve for two disks (source temp) Move disk from A to C (source destination) Solve for two disks (temp destination) Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 18

Towers of Hanoi: Solve Recursively! cont’d o We can solve the puzzle for n disks if we already know how to solve it for n-1 disks. o Solve n disks (source = A, destination = C) n n n Solve for n-1 disks (source temp) Move disk from A to C (source destination) Solve for n-1 disks (temp destination) Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 19

Towers of Hanoi: Recursive Solution private static final char A = 'A'; // initial source private static final char B = 'B'; // initial temp private static final char C = 'C'; // initial destination private static int count = 0; private static void move(char from, char to) { System. out. printf("%2 d: Move disk from %c to %c. n", ++count, from, to); } public static void main(String args[]) { int n = 6; System. out. printf("Solve for %d disks: nn", n); solve(n, A, C, B); Hanoi 1. java } Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 20

Towers of Hanoi: Solve Recursively! o Solve n disks (source = A, destination = C) n n n Solve for n-1 disks (source temp) Move disk from A to C (source destination) Solve for n-1 disks (temp destination) private static void solve(int n, char source, char destination, char temp) { if (n > 0) { solve(n-1, source, temp, destination); move(source, destination); solve(n-1, temp, destination, source); } Hanoi 1. java } Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak Demo 21

Towers of Hanoi: Analysis o How can we measure how long it will take to solve the puzzle for n disks? o What’s a good predictor? n n The number times we move a disk from one pin to another. Therefore, let’s count the number of moves. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 22

Towers of Hanoi: Analysis, cont’d o Solve n disks (source = A, destination = C) n n n o Solve for n-1 disks (source temp) Move disk from A to C (source destination) Solve for n-1 disks (temp destination) What is the pattern in the number of moves as n increases? n Let f(n) be the number of moves for n disks. { f(n) = Computer Science Dept. Summer 2015: June 2 1 n=1 2 f(n-1) + 1 n > 1 CS 146: Data Structures and Algorithms © R. Mak 23

Towers of Hanoi: Analysis { f(n) = o This is a recurrence relation. n n o 1 n=1 2 f(n-1) + 1 n > 1 f shows up in its own definition: f(n) = 2 f(n-1) + 1 The mathematical analogy of recursion. Can we find the definition of function f ? n n Observation: Since f(n) = 2 f(n-1) + 1, we know that f(n) > 2 f(n-1). Therefore, if we increase the number of disks from n to n+1, the number of moves will at least double. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 24

Towers of Hanoi: Count Moves private static void move(char from, char to) { ++count; Don’t print. Just count moves. } public static void main(String args[]) { System. out. println("Disks Moves"); for (int n = 1; n <= 10; n++) { count = 0; solve(n, A, C, B); System. out. printf("%5 d %5 dn", n, count); } Hanoi 2. java } Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak Demo 25

Towers of Hanoi: Analysis Disks Moves 1 1 2 3 3 7 4 15 5 31 6 63 7 127 8 255 9 511 10 1023 Computer Science Dept. Summer 2015: June 2 o What’s the pattern? f(n) = 2 n - 1 o Can we prove this? n Just because this formula holds for the first 10 values of n, does it hold for all values of n ≥ 1? CS 146: Data Structures and Algorithms © R. Mak 26

Proof by Induction: Base Case Prove that if: { f(n) = o o 1 n=1 2 f(n-1) + 1 then: n>1 f(n) = 2 n - 1 for all n ≥ 1 Let n = 1. Then f(1) = 21 - 1 = 1 is true. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 27

Proof by Induction: Inductive Step Prove that if: { f(n) = o o o 1 n=1 2 f(n-1) + 1 then: n>1 f(n) = 2 n - 1 for all n ≥ 1 Let n > 1. Inductive hypothesis: Assume that f(k) = 2 k - 1 is true for all k < n, where n > 1 Since n-1 < n, then by our hypothesis: f(n-1) = 2 n-1 – 1. From the recurrence relation: f(n) = 2 f(n-1) + 1 = 2(2 n-1 - 1) + 1 = 2 n -1. So if f(k) = 2 k - 1 is true for all k < n, it must also be true for n as well. Therefore, f(n) = 2 n -1 for all n > 1. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 28

Break Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 29

Algorithm Analysis o An algorithm is a set of operations to perform in order to solve a problem. o We want to know how an algorithm scales as its input size grows. o If T(N) is the running time of an algorithm with N input values, then how does T(N) change as N increases? Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 30

Big-Oh and its Cousins o Let T(N) be the running time of an algorithm with N input values. o Big-Oh n T(N) = O(f(N)) if there are positive constants c and n 0 such that T(N) ≤ cf(N) when N ≥ n 0. n In other words, when N is sufficiently large, function f(N) is an upper bound for time function T(N). o We don’t care about small values of N. n T(N) will grow no faster than f(N) as N increases. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 31

Big-Oh and its Cousins, cont’d o Let T(N) be the running time of an algorithm with N input values. o Omega n T(N) = Ω(g(N)) if there are positive constants c and n 0 such that T(N) ≥ cg(N) when N ≥ n 0. n In other words, when N is sufficiently large, function g(N) is lower bound for time function T(N). o We don’t care about small values of N. n T(N) will grow at least as fast as g(N) as N increases. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 32

Big-Oh and its Cousins, cont’d cf(N) T(N) cg(N) N N T(N) = O(f(N)) Upper bound Computer Science Dept. Summer 2015: June 2 T(N) = Ω(g(N)) Lower bound CS 146: Data Structures and Algorithms © R. Mak 33

Big-Oh and its Cousins, cont’d o Let T(N) be the running time of an algorithm with N input values. o Theta n T(N) = Θ(h(N)) if and only if: o o n T(N) = O(h(N)) and T(N) = Ω(h(N)) In other words, the rate of growth of T(N) equals the rate of growth of h(N). Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 34

Big-Oh and its Cousins, cont’d o Let T(N) be the running time of an algorithm with N input values. o Little-Oh n T(N) = o(p(N)) if there are positive constants c and n 0 such that T(N) < cp(N) when N ≥ n 0. n p(N) is similar to the upper bound function f(N) but instead of T(N) ≤ cf(N) we have T(N) < cp(N). Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 35

Big-Oh and its Cousins, cont’d o If T 1(N) = O(f 1(N)) and T 2(N) = O(f 2(N)) then n n o If T(N) is a polynomial of degree k, then n o T 1(N) + T 2(N) = O(f 1(N) + f 2(N)) or O(max(f 1(N), f 2(N))) T 1(N) x T 2(N) = O(f 1(N) x f 2(N)) T(N) = Θ(Nk) If T(N) = logk N for any constant k, then n n T(N) = O(N) Logarithms grow slowly! Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 36

Towers of Hanoi: Rate of Growth o We decided that a good predictor of T(n) for solving the Towers of Hanoi problem was f(n). n n o n is the number of disks f(n) = 2 n-1 is the number of disk moves Therefore, T(n) = Θ(2 n) Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 37

Compare Growth Rates o If we want to compare the growth rates of two functions f(N) and g(N), compute lim f(N) / g(N) N ∞ n The limit is 0: f(N) = o(g(N)) g(N) is an upper bound for f(N). n The limit is a constant c ≠ 0: f(N) = Θ(g(N)) f(N) and g(N) have the same growth rate. n The limit is ∞: g(N) = o(f(N)) f(N) is an upper bound for g(N). Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 38

General Rules for Computing Running Time o Consecutive statements n n o Add the running times of the statements. Generally, only consider the statement with the maximum running time. Branching statement n The running time of the entire statement is at most the maximum running time of its branches. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 39

Computing Running Time, cont’d o Loop n o The running time of a loop is at most the number of iterations times the running time of the statements in the loop. Nested loops n Compute the running time of the statements in the innermost loop, then multiply by the product of the numbers of iterations of all the loops. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 40

Scalability of Different Algorithms o Problem: Given an array of positive and negative integers, find the maximum sum of a contiguous subsequence of the array. o Four algorithms to solve this problem: n n Linear. Runtime. Growth: Logarithmic. Runtime. Growth: Quadratic. Runtime. Growth: Cubic. Runtime. Growth: T(N) = O(N) T(N) = O(N log N) T(N) = O(N 2) T(N) = O(N 3) Demo Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 41

Scalability of Different Algorithms, cont’d o One set of results for the maximum sum problem. n Times in milliseconds. Max. Subseq 2. java n 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Linear 1 1 0 0 0 0 0 Computer Science Dept. Summer 2015: June 2 Logarithmic 0 2 0 0 0 0 Quadratic 4 1 2 5 6 9 12 16 20 24 CS 146: Data Structures and Algorithms © R. Mak Cubic 120 890 1467 3436 6698 11392 18344 27235 39085 53218 42

Scalability of Different Algorithms, cont’d o Problem: Compute the nth Fibonacci number. o Two algorithms to solve this problem: n Start with 1, 1, and repeatedly add the previous two values. o n Linear. Growth. Rate: T(N) = O(N) Use recursion: fib(n) = fib(n-2) + fib(n-1) o Exponential. Growth. Rate: Why is the growth rate exponential? T(N) = Ω(1. 5 N) Demo Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 43

Scalability of Different Algorithms, cont’d o One set of results for the Fibonacci problem. n Times in milliseconds. n 5 10 15 20 25 30 35 40 45 50 Computer Science Dept. Summer 2015: June 2 Linear 0 0 0 0 0 Fibonacci 2. java Exponential 0 1 0 2 3 4 45 504 5358 59267 CS 146: Data Structures and Algorithms © R. Mak 44

Assignment #1: Text Search o Download the complete text of War and Peace as an ASCII file from http: //www. cs. sjsu. edu/~mak/CS 146/assignmen ts/1/War. And. Peace. txt n n Over 65, 000 lines. Over a half million words. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 45

Assignment #1: Text Search o Write a Java program to search for the following names in the text: n n n Makar Alexeevich Joseph Bazdeev Boris Drubetskoy Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 46

Assignment #1: Text Search, cont’d o For each occurrence of each name, print n n n o the starting line number (first line is 1) the starting character position (first position is 1) the name Example output: LINE 19949 21953 22173 46612 Computer Science Dept. Summer 2015: June 2 POSITION 1 2 9 19 NAME Boris Drubetskoy Makar Alexeevich Boris Drubetskoy Joseph Bazdeev CS 146: Data Structures and Algorithms © R. Mak 47

Assignment #1: Text Search, cont’d o Notes n n n o A name can be split across two consecutive lines. More than one name can be on a line. You must print the names in the order that they appear in the text. Print how long your program runs: long start = System. current. Time. Millis(); /* do everything here */ long elapsed = System. current. Time. Millis() - start; n Run your program several times and pick the median time. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 48

Assignment #1: Text Search, cont’d o You may work individually as a team of one, or you can partner with another student as a team of two. n n n You can be on only one team at a time. If you partner with someone, both of you will receive the same score for this assignment. You’ll be able to choose a different partner or work alone for subsequent assignments. Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 49

Assignment #1: Text Search, cont’d o Create a zip file containing: n n Your Java source files. A sample output file. o n o Use output redirection, or cut-and-paste into a text file. A short report (at most 3 pages) describing your algorithm and how well it will scale with respect to the length of the text and the number and lengths of the names to search. Name the zip file after yourself or yourselves. Examples: smith. zip, smith-jones. zip Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 50

Assignment #1: Text Search, cont’d o Email the zip file to ron. mak@sjsu. edu o The subject line must be: CS 146 Assignment #1 Your name(s) n Example: CS 146 Assignment #1 Mary Smith & John Jones o Due: Monday, June 8 at 11: 59 PM Computer Science Dept. Summer 2015: June 2 CS 146: Data Structures and Algorithms © R. Mak 51