Applied Computing Lab Data Structures ChuanMing Liu Computer

Applied Computing Lab Data Structures Chuan-Ming Liu Computer Science & Information Engineering National Taipei University of Technology Taiwan 1 CSIE, NTUT, TAIWAN

Applied Computing Lab Instructor Chuan-Ming Liu (劉傳銘) Office: 1530 Technology Building Computer Science and Information Engineering National Taipei University of Technology TAIWAN Phone: (02) 2771 -2171 ext. 4251 Email: cmliu@csie. ntut. edu. tw Office Hours: Mon: 11: 10 -12: 00, 13: 10 -14: 00 and Thu: 10 - 12: 00, OR by appointment. 2 CSIE, NTUT, TAIWAN

Applied Computing Lab Teaching Assisant Bill In-Chi Su (蘇英啟) Office: 1226 Technology Building Office Hours: Tue: 10: 00~12: 00 and Wed: 10: 00 ~ 12: 00, OR by appointment Email: billandcs@gmail. com Phone: 02 -2771 -2171 ext. 4262 3 CSIE, NTUT, TAIWAN

Applied Computing Lab Text Books Ellis Horowitz, Sartaj Sahni, and Susan Anderson-Frees, Fundamentals of Data Structures in C, 2 nd edition, Silicon Press, 2008. • • • Supplementary Texts Michael T. Goodrich and Roberto Tamassia, Data Structures and Algorithms in JAVA, 4 th edition, John Wiley & Sons, 2006. ISBN: 0471 -73884 -0. Sartaj Sahni, Data Structures, Algorithms, and Applications in JAVA, 2 nd edition, Silicon Press, 2005. ISBN: 0 -929306 -33 -3. Frank M. Carranno and Walter Savitch, Data Structures and Abstractions with Java, Prentice Hall, 2003. ISBN: 0 -13 -017489 -0. 4 CSIE, NTUT, TAIWAN

Applied Computing Lab Course Outline • • • Introduction and Recursion Analysis Tools Arrays Stacks and Queues Linked Lists Sorting Hashing Trees Priority Queues Search Trees Graphs CSIE, NTUT, TAIWAN 5

Applied Computing Lab Course Work • Assignments (50%): 6 -8 homework sets • Midterm (20%) • Final exam (30%) 6 CSIE, NTUT, TAIWAN

Applied Computing Lab Course Policy (1) • No late homework is acceptable. • For a regrade please contact me for the question within 10 days from the date when the quiz or exam was officially returned. No regrading after this period. • Cheating directly affects the reputation of the Department and the University and lowers the morale of other students. Cheating in homework and exam will not be tolerated. An automatic grade of 0 will be assigned to any student caught cheating. Presenting another person's work as your own constitutes cheating. Everything you turn in must be your own doing. 7 CSIE, NTUT, TAIWAN

Applied Computing Lab Course Policy (2) • The following activities are specifically forbidden on all graded course work: – Theft or possession of another student's solution or partial solution in any form (electronic, handwritten, or printed). – Giving a solution or partial solution to another student, even with the explicit understanding that it will not be copied. – Working together to develop a single solution and then turning in copies of that solution (or modifications) under multiple names. 8 CSIE, NTUT, TAIWAN

Applied Computing Lab First Thing to Do • Please visit the course web site http: //www. cc. ntut. edu. tw/~cmliu/DS/NTUT_ DS_S 09 u-GIT/ • Send an email to me using the email address: cmliu@csie. ntut. edu. tw. I will make a mailing list for this course. All the announcements will be broadcast via this mailing list. 9 CSIE, NTUT, TAIWAN

Applied Computing Lab Introduction Chuan-Ming Liu Computer Science & Information Engineering National Taipei University of Technology Taiwan 10 CSIE, NTUT, TAIWAN

Applied Computing Lab Outline • Data Structures and Algorithms • Pseudo-code • Recursion 11 CSIE, NTUT, TAIWAN

Applied Computing Lab What is Data Structures • A data structure * in computer science is a way of storing data in a computer so that it can be used efficiently. – An organization of mathematical and logical concepts of data – Implementation using a programming language – A proper data structure can make the algorithm or solution more efficient in terms of time and space * Wikipedia: http: //en. wikipedia. org/wiki/Data_structure 12 CSIE, NTUT, TAIWAN

Applied Computing Lab Why We Learn Data Structures • Knowing data structures well can make our programs or algorithms more efficient • In this course, we will learn – Some basic data structures – How to tell if the data structures are good or bad – The ability to create some new and advanced data structures 13 CSIE, NTUT, TAIWAN

Applied Computing Lab What is an Algorithm (1) An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. All the algorithms must satisfy the following criteria: – Input – Output – Precision (Definiteness) – Effectiveness – Finiteness 14 CSIE, NTUT, TAIWAN

Applied Computing Lab What is an Algorithm (2) • Definiteness: each instruction is clear and unambiguous • Effectiveness: each instruction is executable; in other words, feasibility • Finiteness: the algorithm terminates after a finite number of steps. 15 CSIE, NTUT, TAIWAN

Applied Computing Lab What is an Algorithm (3) Input Definiteness Effectiveness Finiteness Output Computational Procedures 16 CSIE, NTUT, TAIWAN

Applied Computing Lab Procedures vs. Algorithms • Termination or not • One example for procedure is OS • Program, a way to express an algorithm 17 CSIE, NTUT, TAIWAN

Applied Computing Lab Expressing Algorithms • Ways to express an algorithm – Graphic (flow chart) – Programming languages (C/C++) – Pseudo-code representation 18 CSIE, NTUT, TAIWAN

Applied Computing Lab Outline • Data Structures and Algorithms • Pseudo-code • Recursion 19 CSIE, NTUT, TAIWAN

Applied Computing Lab Example – Selection Sort Suppose we must devise a program that sorts a collection of n 1 elements. Idea: Among the unsorted elements, select the smallest one and place it next in the sorted list. for (int i=1; i<=n; i++) { examine a[i] to a[n] and suppose the smallest element is at a[j]; interchange a[i] and a[j]; } 20 CSIE, NTUT, TAIWAN

Applied Computing Lab Pseudo-code for Selection Sort Selection. Sort(A) /* Sort the array A[1: n] into nondecreasing order. */ for i 1 to length[A] do j i for k (i+1) to length[A] do if A[k]<A[j] then j k; t A[i] A[j] t 21 CSIE, NTUT, TAIWAN

Applied Computing Lab Maximum of Three Numbers This algorithm finds the largest of the numbers a, b, and c. Input Parameters: a, b, c Output Parameter: x max(a, b, c, x) { x=a if (b > x) // if b is larger than x, update x x=b if (c > x) // if c is larger than x, update x x=c } CSIE, NTUT, TAIWAN 22

Applied Computing Lab Pseudo-Code Conventions • Indentation as block structure • Loop and conditional constructs similar to those in PASCAL, such as while, for, repeat( do – while) , if-then-else • // as the comment in a line • Using = for the assignment operator • Variables local to the given procedure 23 CSIE, NTUT, TAIWAN

Applied Computing Lab Pseudo-Code Conventions • Relational operators: ==, != , , . • Logical operators: &&, ||, !. • Array element accessed by A[i] and A[1. . j] as the subarray of A 24 CSIE, NTUT, TAIWAN

Applied Computing Lab Outline • Data Structures and Algorithms • Pseudo-code • Recursion 25 CSIE, NTUT, TAIWAN

Applied Computing Lab Recursion • Recursion is the concept of defining a method that makes a call to itself • A method calling itself is making a recursive call • A method M is recursive if it calls itself (direct recursion) or another method that ultimately leads to a call back to M (indirect recursion) 26 CSIE, NTUT, TAIWAN

Applied Computing Lab Repetition • Repetition can be achieved by – Loops (iterative) : for loops and while loops – Recursion (recursive) : a function calls itself • Factorial function – General definition: n! = 1· 2· 3· ··· · (n-1)· n – Recursive definition 27 CSIE, NTUT, TAIWAN

Applied Computing Lab Method for Factorial Function • recursive factorial function public static int recursive. Factorial(int n) { if (n == 0) return 1; else return n * recursive. Factorial(n- 1); } 28 CSIE, NTUT, TAIWAN

Applied Computing Lab Content of a Recursive Method • Base case(s) – Values of the input variables for which we perform no recursive calls are called base cases (there should be at least one base case). – Every possible chain of recursive calls must eventually reach a base case. • Recursive calls – Calls to the current method. – Each recursive call should be defined so that it makes progress towards a base case. 29 CSIE, NTUT, TAIWAN

Applied Computing Lab Visualizing Recursion Example recursion trace: • Recursion trace • A box for each recursive call • An arrow from each caller to callee • An arrow from each callee to caller showing return value return 4*6 recursive. Factorial(4) return 3*2 recursive. Factorial(3) return 2*1 recursive. Factorial(2) return 1*1 recursive. Factorial(1) return 1 recursive. Factorial(0) 30 CSIE, NTUT, TAIWAN

Applied Computing Lab About Recursion • Advantages – Avoiding complex case analysis – Avoiding nested loops – Leading to a readable algorithm description – Efficiency • Examples – File-system directories – Syntax in modern programming languages 31 CSIE, NTUT, TAIWAN

Applied Computing Lab Linear Recursion • The simplest form of recursion • A method M is defined as linear recursion if it makes at most one recursive call • Example: Summing the Elements of an Array – Given: An integer array A of size m and an integer n, where m ≧n≧ 1. – Problem: the sum of the first n integers in A. 32 CSIE, NTUT, TAIWAN

Applied Computing Lab Summing the Elements of an Array • Solutions: – using (for) loop Algorithm Linear. Sum(A, n): Input: integer array A, an integer n ≧ 1, such that A has at least n elements – using recursion Output: The sum of the first n integers in A if n = 1 then return A[0] else return Linear. Sum(A, n - 1) + A[n - 1] Note: the index starts from 0. CSIE, NTUT, TAIWAN 33

Applied Computing Lab Recursive Method • An important property of a recursive method – the method terminates • An algorithm using linear recursion has the following form: – Test for base cases – Recur 34 CSIE, NTUT, TAIWAN

Applied Computing Lab Analyzing Recursive Algorithms • Recursion trace – Box for each instance of the method – Label the box with parameters – Arrows for calls and returns call return 15 + A[4] = 15 + 5 = 20 Linear. Sum (A, 5) call return 13 + A[3] = 13 + 2 = 15 Linear. Sum (A, 4) call return 7 + A[2] = 7 + 6 = 13 Linear. Sum (A, 3) call return 4 + A[1] = 4 + 3 = 7 Linear. Sum (A, 2) call return A[0] = 4 Linear. Sum (A, 1) 35 CSIE, NTUT, TAIWAN

Applied Computing Lab Example • Reversing an Array by Recursion – Given: An array A of size n – Problem: Reverse the elements of A (the first element becomes the last one, …) • Solutions – Nested loop ? – Recursion 36 CSIE, NTUT, TAIWAN

Applied Computing Lab Reversing an Array Algorithm Reverse. Array(A, i, j): Input: An array A and nonnegative integer indices i and j Output: The reversal of the elements in A starting at index i and ending at j if i < j then Swap A[i] and A[ j] Reverse. Array(A, i + 1, j - 1) return 37 CSIE, NTUT, TAIWAN

Applied Computing Lab Facilitating Recursion • In creating recursive methods, it is important to define the methods in ways that facilitate recursion. • This sometimes requires we define additional parameters that are passed to the method. – For example, we defined the array reversal method as Reverse. Array(A, i, j), not Reverse. Array(A). 38 CSIE, NTUT, TAIWAN

Applied Computing Lab Example – Computing Powers • The power function, p(x, n)=xn, can be defined recursively: • Following the definition leads to an O(n) time recursive algorithm (for we make n recursive calls). • We can do better than this, however. 39 CSIE, NTUT, TAIWAN

Applied Computing Lab Recursive Squaring • We can derive a more efficient linearly recursive algorithm by using repeated squaring: • For example, 24 25 26 27 = 2(4/2)2 = (22)2 = 42 = 16 = 21+(4/2)2 = 2(22)2 = 2(42) = 32 = 2(6/2)2 = (23)2 = 82 = 64 = 21+(6/2)2 = 2(23)2 = 2(82) = 128 CSIE, NTUT, TAIWAN 40

Applied Computing Lab A Recursive Squaring Method Algorithm Power(x, n): Input: A number x and integer n ≧ 0 Output: The value xn if n = 0 then return 1 if n is odd then y = Power(x, (n - 1)/ 2) return x · y ·y else /* n is even */ y = Power(x, n/ 2) return y · y 41 CSIE, NTUT, TAIWAN

Analyzing the Recursive Squaring Method Applied Computing Lab Algorithm Power(x, n): Input: A number x and integer n ≧ 0 Output: The value xn if n = 0 then return 1 if n is odd then y = Power(x, (n - 1)/ 2) return x · y else y = Power(x, n/ 2) return y · y Each time we make a recursive call we halve the value of n; hence, we make log n recursive calls. That is, this method runs in O(log n) time. It is important that we used a variable twice here rather than calling the method twice. 42 CSIE, NTUT, TAIWAN

Applied Computing Lab Tail Recursion • Tail recursion occurs when a linearly recursive method makes its recursive call as its last step. • Such methods can be easily converted to non-recursive methods (which saves on some resources). • The array reversal method is an example. 43 CSIE, NTUT, TAIWAN

Applied Computing Lab Example – Using Iteration Algorithm Iterative. Reverse. Array(A, i, j ): Input: An array A and nonnegative integer indices i and j Output: The reversal of the elements in A starting at index i and ending at j while i < j do Swap A[i ] and A[ j ] i =i+1 j =j-1 return 44 CSIE, NTUT, TAIWAN

Applied Computing Lab Binary Recursion • Binary recursion occurs whenever there are two recursive calls for each non-base case. • Example: Binay. Sum 45 CSIE, NTUT, TAIWAN

Example – Summing n Elements in an Array Applied Computing Lab • Recall that this problem has been solved using linear recursion • Using binary recursion instead of linear recursion Algorithm Binary. Sum(A, i, n): Input: An array A and integers i and n Output: The sum of the n integers in A starting at index i if n = 1 then return A[i ] return Binary. Sum(A, i, n/2 )+Binary. Sum(A, i+ n/2 , n/2 ) 46 CSIE, NTUT, TAIWAN

Applied Computing Lab Recursion Trace 0, 8 0, 4 4, 4 0, 2 0, 1 2, 2 1, 1 2, 1 4, 2 3, 1 4, 1 6, 2 5, 1 6, 1 7, 1 • Note the floor and ceiling used in the method 47 CSIE, NTUT, TAIWAN

Applied Computing Lab Fibonacci Numbers • Fibonacci numbers are defined recursively: F 0 = 0 F 1 = 1 Fi = Fi-1 + Fi-2 for i > 1. • Example: 0, 1, 1, 2, 3, 5, 8, … 48 CSIE, NTUT, TAIWAN

Fibonacci Numbers – Binary Recursion Applied Computing Lab Algorithm Binary. Fib(k): Input: Nonnegative integer k Output: The kth Fibonacci number Fk if k ≦ 1 then return k else return Binary. Fib(k-1) + Binary. Fib(k-2) 49 CSIE, NTUT, TAIWAN

Applied Computing Lab Analyzing the Binary Recursion • Algorithm Binary. Fib makes a number of calls that are exponential in k • By observation, there are many redundant computations: F 0 = 0; F 1 = 1; F 2 = F 1 + F 0; F 3 = F 2 + F 1 =(F 1 + F 0)+ F 1; … • The above two results show the inefficiency of the method using binary recursion 50 CSIE, NTUT, TAIWAN

Applied Computing Lab A Better Fibonacci Algorithm • Using linear recursion instead – avoid the redundant computation: Algorithm Linear. Fibonacci(k): Input: A nonnegative integer k Output: Pair of Fibonacci numbers (Fk, Fk-1) if k = 1 then return (k, 0) else (i, j) = Linear. Fibonacci(k - 1) return (i +j, i) • Runs in O(k) time. 51 CSIE, NTUT, TAIWAN

Applied Computing Lab Multiple Recursion • Motivating example: summation puzzles • pot + pan = bib • dog + cat = pig • boy + girl = baby • Multiple recursion: makes potentially many recursive calls (not just one or two). 52 CSIE, NTUT, TAIWAN

Applied Computing Lab Algorithm for Multiple Recursion Algorithm Puzzle. Solve(k, S, U): Input: An integer k, sequence S, and set U (the universe of elements to test) Output: An enumeration of all k-length extensions to S using elements in U without repetitions for all e in U do Remove e from U {e is now being used} Add e to the end of S if k = 1 then Test whether S is a configuration that solves the puzzle if S solves the puzzle then return “Solution found: ” S else Puzzle. Solve(k - 1, S, U) Add e back to U {e is now unused} Remove e from the end of S 53 CSIE, NTUT, TAIWAN

Applied Computing Lab Visualizing Puzzle. Solve() Initial call Puzzle. Solve (3, (), {a, b, c}) Puzzle. Solve (2, b, {a, c}) Puzzle. Solve (2, a, {b, c}) Puzzle. Solve (1, ab , {c}) Puzzle. Solve (1, ba , {c}) abc bac Puzzle. Solve (2, c, {a, b}) Puzzle. Solve (1, ca , {b}) cab Puzzle. Solve (1, ac , {b}) Puzzle. Solve (1, bc , {a}) Puzzle. Solve (1, cb , {a}) acb bca cba 54 CSIE, NTUT, TAIWAN