CS 146 Data Structures and Algorithms July 7

CS 146: Data Structures and Algorithms July 7 Class Meeting Department of Computer Science San Jose State University Summer 2015 Instructor: Ron Mak www. cs. sjsu. edu/~mak

Midterm Solutions 1. Suppose variable p contains a reference to a node in a singly linked list. Each list node has a data field and a next field that refers to the next node in the list, and you can access both these fields. You are told that p does not refer to the last node of the list. You do not have a reference to the head node of the list. Describe in words or in Java pseudocode how you can delete the node referred to by p and yet maintain the integrity of the list. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 2

Midterm Solutions, cont’d A B C D E F p Problem: You cannot simply delete node C because you can’t access node B in order to get it to point to node D. Solution: n Copy the data field of node D into the data field of node C. n Now point node C to node D’s successor (node E). Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 3

Midterm Solutions, cont’d 2. Use Big-Oh notation to describe the number of times with respect to N that the variable sum is incremented in of each of the following program fragments. a. sum = 0; for (n = N; n > 0; n /= 2) { for (i = 0; i < n; i++) { sum++; } } Answer: The inner loop goes N + N/2 + N/4 + N/8 + … times. Therefore, O(N). Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 4

Midterm Solutions, cont’d b. sum = 0; for (i = 1; i < N; i *= 2) { for (j = 0; j < i; j++) { sum++; } } c. sum = 0; for (i = 1; i < N; i *= 2) { for (j = 0; j < N; j++) { sum++; } } Computer Science Dept. Summer 2015: July 7 Answer: The inner loop goes 1 + 2 + 4 + 8 + … + N times. Therefore, O(N) Answer: The inner loop always goes N times. The outer loop goes log 2 N times. Therefore, O(N log 2 N) CS 146: Data Structures and Algorithms © R. Mak 5

Midterm Solutions, cont’d d. sum = 0; for (i = 0; i < N; i++) { for (j = 0; j < i*i; j++) { for (k = 0; k < j; k++) { sum++; } } } e. sum = 0; for (i = 1; i < for (j = 1; if (j%i for } } Answer: j can be as large as i 2, which could be as large as N 2. k can be as large as j, which can be as large as N 2. Therefore, the count is proportional to N N 2, which is O(N 5) N; i++) { j < i*i; j++) { == 0) { (k = 0; k < j; k++) { sum++; Answer: Without the if statement, the answer would be O(N 5) as above. But the if statement is true only i times as j loops i 2. times. } Therefore, O(N 4). } Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 6

Midterm Solutions, cont’d 3. Write in Java or pseudocode a method that, when passed the root of an arbitrary binary tree, converts the tree to its mirror and returns the root of the mirror: Solution: n n Do a preorder traversal of the tree. Swap the left and right children of each node that you visit. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 7

Midterm Solutions, cont’d 4. Draw step-by-step the insertion of the node values 37, 54, 65, 49, 39, 28, and 59 into an initially empty AVL tree. Identify any rotations along the way. Inserting node 37: Inserting node 65: 37 Single left rotation: 54 Inserting node 54: 37 54 / / 37 65 54 Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 8

Midterm Solutions, cont’d Inserting node 28: Inserting node 49: Single right rotation: 39 54 / -- 39 / -- / 37 65 -/ 37 / 49 / 28 Inserting node 39: Double right-left rotation: 39 54 / -/ 39 / / 37 - 65 49 Computer Science Dept. Summer 2015: July 7 - 54 / / 49 65 Inserting node 59: 39 / ------/ 37 54 / / --- -/ / 28 49 65 / / 59 CS 146: Data Structures and Algorithms © R. Mak 9

Midterm Solutions, cont’d 5. Consider an N-by-N matrix of numbers. a. Suppose the numbers in the matrix are in a random order. Using Big-Oh notation in terms of N, how long would it take to search the matrix for a particular value? Answer: Up to N 2 numbers have to be checked, so O(N 2). b. Now suppose the numbers in the matrix are in sequential order. Describe in words an algorithm to search for a particular value, and how long would it take in terms of N? Answer: Start with the upper left corner of the matrix, and go either down or to the right while checking each number. You will go down at most N rows and to the right at most N columns to check at most 2 N numbers. Therefore, O(N). Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 10

Midterm Solutions, cont’d 6. You are given two unsorted arrays of numbers, each with N elements. a. Describe in words an algorithm that successively finds the smallest N pairs of elements, with each pair containing an element from each array. The first pair contains the smallest value from each array, the second pair contains the second smallest value from each array, etc. each array into a heap. Then successively Answer: First convert remove the first element of each heap to form the pairs. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 11

Midterm Solutions, cont’d b. Using Big-Oh in terms of N, how long will your algorithm take to find all N pairs? Answer: It takes O(N) time to build a heap from an array (method build. Heap()). It takes O(log N) to remove a single value from a heap, therefore O(N log N) to remove N values. Altogether: O(N log N) to find all N pairs. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 12

Midterm Solutions, cont’d 7. Consider the following recurrence relation for n that are exact powers of 2: T(n) = a. { 2 2 T(n/2) + n if n = 2 k, for k > 1 What is T(4)? Answer: T(4) = 2 T(2) + 4 = 4 + 4 = 8 T(16)? Answer: T(8) = 2 T(4) + 8 = 16 + 8 = 24 T(16) = 2 T(8) + 16 = 48 + 16 = 64 Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 13

Midterm Solutions, cont’d T(n) = b. { 2 2 T(n/2) + n if n = 2 k, for k > 1 Use mathematical induction to prove that the solution of the recurrence relation is T(n) = n(log 2 n). Base case: n = 2 T(2) = 2(log 22) = 2(1) = 2 is true Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 14

Midterm Solutions, cont’d T(n) = { 2 2 T(n/2) + n if n = 2 k, for k > 1 Prove that T(n) = n(log 2 n) Induction hypothesis: Assume it’s true for n = 2 k-1: On both sides: Multiply by 2 and add 2 k On the left: Apply the recurrence relation Therefore, if Computer Science Dept. Summer 2015: July 7 On the right: factor then CS 146: Data Structures and Algorithms © R. Mak 15

A Solution to Assignment #2 get(i), k = 100 N 100000 Array. List 0 0 Linked. List Indexed. List 0 0 5 3 47 4 4071 4 add(x, i), k = 100 N 100000 Computer Science Dept. Summer 2015: July 7 Array. List 0 0 8 298 Linked. List Indexed. List 0 0 1 1 29 9 4633 60 CS 146: Data Structures and Algorithms © R. Mak 16

A Solution to Assignment #2, cont’d remove(i), k = 100 N 100000 Array. List 0 0 5 302 Linked. List Indexed. List 0 0 1 0 23 8 2199 65 Mixture, k = 100 N 100000 Computer Science Dept. Summer 2015: July 7 Array. List 0 0 3 239 Linked. List Indexed. List 0 0 1 0 30 1 4082 46 CS 146: Data Structures and Algorithms © R. Mak 17

A Solution to Assignment #2, cont’d Indexed list, N = 100000 Operation get(i) add(x, i) remove(i) mixture o k=2 2 1883 2095 1390 k=10 2 435 438 359 k=100 4 51 48 48 k=10000 38 536 66 579 38 309 46 363 This is why I chose k = 100 for my other tests. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 18

public Integer get(int i) { return get. Node(i). data(); } private Node get. Node(int i) { x. Position = i/k; x. Delta = i - k*x. Position; Get the position in the index array and compute how far we are from the “indexed node” to the desired node. If more than halfway forward from the if (x. Delta > k/2) { indexed node, use the next indexed node if (x. Position < index. size()-1) { and go backward instead to the desired node. x. Position++; x. Delta -= k; // now negative to move backward } } Node node = index. get(x. Position); Get the indexed node and move forward or backward to the desired node. if (x. Delta > 0) { int d = x. Delta; while (d-- > 0) node = node. next(); } else { int d = x. Delta; while (d++ < 0) node = node. prev(); } return node; Computer Science Dept. CS 146: Data Structures and Algorithms } Summer 2015: July 7 © R. Mak 19

A Solution to Assignment #2, cont’d public void add(int i, Integer data) { Node node. After = get. Node(i); Node new. Node = new Node(data); insert(new. Node, node. After); // Initialize the index array if this is // the first hybrid list node. if (N == 1) { index. add(new. Node); x. Residual = 0; } } // Update the index array by pointing backward one node // following the insertion point. else if (x. Delta > 0) { update. Index. Backward(x. Position + 1); Inserted after the indexed node. } else { update. Index. Backward(x. Position); Inserted before the indexed node. } Computer Science Dept. CS 146: Data Structures and Algorithms Summer 2015: July 7 © R. Mak 20

A Solution to Assignment #2, cont’d public Integer remove(int i) { Node node = get. Node(i); delete(node); // Update the index array by pointing forward one node // following the insertion point. if (x. Delta > 0) { update. Index. Forward(x. Position + 1); Deleted after the indexed node. } else { update. Index. Forward(x. Position); Deleted before the indexed node. } return node. data(); } Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 21

A Solution to Assignment #2, cont’d o After each insertion, update the index array by pointing each indexed node to its predecessor. n May need to append a new index array element. private void update. Index. Backward(int pos) { for (int j = pos; j < index. size(); j++) { index. set(j, index. get(j). prev()); } Point each index node to its predecessor. // If the residual is now k, append a new index array element. if (++x. Residual == k) { Append a new index node index. add(tail. prev()); if the residual reaches k. x. Residual = 0; } } Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 22

A Solution to Assignment #2, cont’d o After each deletion, update the index array by pointing each indexed node to its successor. n May need to remove the last index array element. private void update. Index. Forward(int pos) { for (int j = pos; j < index. size(); j++) { index. set(j, index. get(j). next()); } Point each index node to its successor. // If the residual was 0, delete the last index array element. if (x. Residual-- == 0) { index. remove(index. get(index. size()-1)); Remove the last index node if the residual reached 0. x. Residual = k-1; } } Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 23

Break Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 24

Sorting o Convert unordered data into ordered data. n n n o Several algorithms for sorting. Some simple, some very elegant. Many opportunities for analysis. Sorting involves comparing two values and possibly swapping (or moving) them. n n Goal: The most efficient algorithm possible. Minimize the numbers of comparisons and swaps. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 25

Insertion Sort o One of the simplest and intuitive algorithms. n o The way you would manually sort a deck of cards. Make N– 1 passes over the list of data. n For pass p = 1 through N– 1, the algorithm guarantees that the data in positions 0 through p– 1 are already sorted. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 26

Insertion Sort public static <Any. Type extends Comparable<? super Any. Type>> void insertion. Sort(Any. Type[] a) { int j; for (int p = 1; p < a. length; p++) { Any. Type tmp = a[p]; Does this value belong in the sorted part? for (j = p; j > 0 && tmp. compare. To(a[j-1]) < 0; j--) { a[j] = a[j-1]; Slide values in the sorted part of the list } a[j] = tmp; one to the right to make room for a new member of the sorted part. } } o The inner for loop terminates quickly if the tmp value does not need to be inserted too far into the sorted part. n The entire sort finishes quickly if the data is nearly sorted: O(N). Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 27

Analysis of Insertion Sort o Nested loops, each can take N iterations. n o An inversion is an ordered pair of data (a[i], a[j]) from the list where the indices i < j but the values a[i] > a[j]. n o Therefore, insertion sort is O(N 2). The two values are out of order. Assume we’re sorting from lowest to highest. n Example: The list 34 8 64 51 32 21 has the inversions (34, 8), (34, 32), (34, 21), (64, 51), (64, 32), (64, 21), (51, 32), (51, 21), and (32, 21). Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 28

Analysis of Insertion Sort, cont’d o Swapping two adjacent values that out of order removes exactly one inversion from the list. n o A sorted array has no inversions. We can compute bounds on the average running time of insertion sort by computing the average number of inversions in a permutation of values in a list. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 29

Analysis of Insertion Sort, cont’d o What is the average number of inversions? o Suppose list L contains N values. n n It has N(N-1)/2 distinct ordered pairs. Each ordered pair is a potential inversion. o Assume half of the ordered pairs is an inversion. o Therefore, the average number of inversions in each list is N(N-1)/4. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 30

Analysis of Insertion Sort, cont’d o Conclusion: Any sorting algorithm that limits comparisons to adjacent items will cost at least N(N-1)/4 = Ω(N 2) in the average case. o Each swap removes only one inversion, so Ω(N 2) swaps are required. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 31

Insertion Sort: Some Observations o Insertion sort is fast if the array is nearly sorted. n o Otherwise, it must move more values in the list. If we can swap non-adjacent values, we may be able to remove more than one inversion at a time. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 32

Insertion Sort: Some Observations, cont’d o If we can get the array “nearly sorted” as soon as possible, insertion sort can finish the job quickly. o These observations led Donald Shell to invent the Shellsort algorithm in 1959. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 33

Shellsort o Like insertion sort, except we compare values that are h elements apart in the list. n h diminishes after completing a pass, for example, 5, 3, and 1. o The final value of h must be 1, so the final pass is a regular insertion sort. o The previous passes get the array “nearly sorted” quickly. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 34

Shellsort, cont’d o After each pass, the array is said to be hk-sorted. n Examples: 5 -sorted, 3 -sorted, etc. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 35

Shellsort o The effect is to perform an insertion sort on hk independent subarrays during each pass. o Shellsort is also called a diminishing increment sort. n The choice of h values affects how long the sort takes. n Don Knuth suggests reversing the sequence 1, 4, 13, 40, . . . (start at 1 and for each subsequent pass, multiply the previous by 3 and add 1). Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 36

Analysis of Shellsort o Shellsort is difficult to analyze. o Worse case running time is Θ(N 2). o Average case using Knuth’s sequence is O(N 1. 5), which is better than insertion sort’s O(N 2). Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 37

Shellsort public static <Any. Type extends Comparable<? super Any. Type>> void shellsort(Any. Type[] a) { Use the (suboptimal) int j; sequence for h which for (int h = a. length/2; h > 0; h /= 2) { for (int i = h; i < a. length; i++) { Any. Type tmp = a[i]; starts at half the list length and is halved for each subsequent pass. for (j = i; j >= h && tmp. compare. To(a[j-h]) < 0; j -= h) { a[j] = a[j-h]; } a[j] = tmp; } } } Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 38

Assignment #4 o This assignment will give you a closer look at the insertion sort and Shellsort algorithms. o Write a program that will sort integer arrays of sizes 1000 10, 000 and 100, 000 using both sorting algorithms. o Both algorithms should sort the same array, so make a copy of it each time before sorting. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 39

Assignment #4, cont’d o The algorithms should sort, in the lowest to highest (increasing) order: n An unsorted array of unique random numbers in random order. n An array of unique numbers that’s already sorted in increasing order. These values do not have to be random. Your algorithms should not know ahead of time that the array was already sorted. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 40

Assignment #4, cont’d n An array of unique numbers that’s already sorted in reverse (decreasing) order. These values do not have to be random. Your algorithms should not know ahead of time that the array was already sorted in reverse. n An array of all zeroes. Your algorithms should not know ahead of time that all the array elements contain the same value. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 41

Assignment #4, cont’d o Do Shellsort twice: n First use the suboptimal sequence for h which starts at half the length of the array and is halved for each pass. n Then use Knuth’s interval sequence 1, 4, 13, 40, . . . (reversed) for values of h. The first value of h should be as large as possible but under half the array length. o Verify that your arrays are properly sorted! o Generalize your program enough to accommodate other sorting algorithms in the future. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 42

Assignment #4, cont’d o For each sort, your program should output: n How much time it took in milliseconds. How many comparisons it made between two values. How many moves it made of the values. n Examples: n n o o o If you shift five values over one position each, that’s five moves. Two values exchanging places (a swap) is two moves. You can output these results in a single table. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 43

Assignment #4, cont’d o You may choose a partner to work with you on this assignment. n o o Both of you will receive the same score. Email your answers to ron. mak@sjsu. edu n Subject line: CS 146 Assignment #4: Your Name(s) n CC your partner when you email your solution. Due Monday, July 13 at 11: 59 PM. Computer Science Dept. Summer 2015: July 7 CS 146: Data Structures and Algorithms © R. Mak 44

Assignment #4 Sample Output ===== Unsorted Random Unique ===== N = 100 ALGORITHM MILLISECONDS Insertion sort 1 Shellsort suboptimal 0 Shellsort Knuth 0 MOVES COMPARES 2, 723 2, 724 590 826 627 718 MOVES COMPARES 247, 291 247, 293 11, 857 15, 339 13, 322 14, 922 MOVES COMPARES 24, 986, 430 24, 986, 435 N = 1, 000 ALGORITHM MILLISECONDS Insertion sort 9 Shellsort suboptimal 1 Shellsort Knuth 1 N = 10, 000 ALGORITHM MILLISECONDS Insertion sort 74 Computer. Shellsort Science Dept. suboptimal 10 Summer 2015: July 7 Shellsort Knuth 275, 513 CS 146: 219, 431 Data Structures and Algorithms © R. Mak 206, 966 228, 206 45