Data Structure Final Review 27 Apr2009 SUNY Buffalo

Data Structure – Final Review 27 -Apr-2009 SUNY Buffalo 1

About this review n I’ve been asked to review several data structures covered in class. n n I’ve used a different book than the one in this class. n n May not be totally complete as it is unrealistic to cover all the materials in ~40 mins! Exam may ask questions that weren’t covered in this review but were covered in class. If you have questions, ask your instructor ASAP. Materials were mostly from “Data structure with C”. I have years of hands on experiences with data structures/algorithms. n 27 -Apr-2009 If you wonder how data structures are used in the “real world”, ask them. Data Structure Review 2

Review Topics n Tree ADT: n n n Heap, AVL Tree, Red-Black Tree, and 2 -3 Tree (B-tree). For each topic, you should prepare to answer: n n n Dictionary (map) ADT: n n n Hash tables and hash functions. n What is it? How to represent it? What operations does it support? How each operation works? n Graph ADT: n n (? ) Breadth-First Search (BFS), and (? ) Depth-First Search (DFS). n How long each operation takes? n n n 27 -Apr-2009 Data Structure Review Practice your drawing; do as much examples as you can! Best-case, Average-case, and Worst-case. 3

Review: Trees n Terminology: n n n Node structure. Storage: Array, Linked list. Binary tree: n n Binary Heap. Binary search tree (BST): n n B-tree: n AVL and R-B. 2 -3 tree. n n Pre-order (Root, L, R), In-order (L, Root, R), Post-order (L, R, Root), Level-order. Time complexity: n n 27 -Apr-2009 insert(), delete(), search(), sort() and etc. Binary tree-walks: n Types: n Operations: n Representation: n n Size, height, depth (level), link (edge), path. Root, parent, children, sibling, leaves, ancestor, descendant, etc. n Data Structure Review Insertion: O (log n), Searching: O (log n), Deletion: O (log n), Sorting: O (n log n). 4

Binary Tree: Importance of Balance n Binary tree, in general, is useful for implementing many operations: n n n But, the insert() and delete() alter the shape of the tree and can result in an unbalanced tree. n n For examples, search(), successor(), predecessor(), minimum(), maximum(), insert(), and delete() can be achieved in O(h) time, where h is the height of the tree. That is, the average running time of above operations on a balanced tree is h = O(lg n). In the worst case, h = O(n) no better than a linked list! So, we want to correct the imbalance in at most O(lg n) time no complexity overhead. 27 -Apr-2009 Data Structure Review 5

Review: Balanced Trees n To make sure a binary tree is balanced, add a requirement, called the heap property, to the binary tree. n n n Binary heap is commonly use for implementing Priority Queue ADT. Aside, heap could also mean the memory space used for dynamic allocation. To make sure a BST is balanced, add a constrain on the height of BST trees. n 27 -Apr-2009 x The most popular data structures are AVL and Red. Black trees. Data Structure Review 6

Review: Binary Heap n A binary heap extends binary tree data structure and has the following properties: n Each node has a key <greater|less> than or equal to the key of its children. n n n Greater - Max heap; Less - Min heap; The tree is a complete binary tree. A complete binary tree is a binary tree in which every level, except possibly the last, is completely filled, and all nodes are as far left as possible. n 27 -Apr-2009 Longest path is ceiling(lg n) for n nodes. Data Structure Review 7

Heap: Maintaining the Heap Property n n heapify. Up() and heaify. Down() are the key operations for maintaining the heap property in O(lg n) time. How does heapify. Down() work? n n How does heapify. Up() work? n n Given a node i in the heap. For maxheap: A[i] < A[left(i)] or A[i] < A[right(i)], swap A[i] with the largest of A[left(i)] and A[right(i)]. Recurs on that sub-tree. Given a node i in the heap. For maxheap: A[i] > A[parent[i]], swap A[i] with A[parent[i]. Recurs on parent[i]. What about other operations and their running time? n 27 -Apr-2009 delete(), insert(), build. Heap(), heap. Sort(). Data Structure Review 8

Review: AVL Tree n An AVL tree extends BST data structure and include the following property: n Any node in the tree has the height difference between its left and right sub-trees is at most one. x h h-1 T h-1 Size of tree: n Th-2 Th = Th– 1 + Th– 2 + 1 Observe that: n n 27 -Apr-2009 h-2 Th AVL: Adelson-Velsky and Landis, 1962 The smallest AVL tree of depth 1 has 1 node. The smallest AVL tree of depth 2 has 2 nodes. Data Structure Review 9

AVL: Maintaining the AVL Property n n Tree rotation is the key operations for maintaining the AVL property in O(lg n) time. If a node is not balanced, the difference between its children heights is 2. n 4 possible cases with a height difference of 2. x y y x A B (1) 27 -Apr-2009 C A B y x x y C A (2) Data Structure Review C A B B (3) (4) C 10

AVL: Maintaining the AVL Property (2) n Case 1: right. Rotate (y); n n x = y. get. Left. Child(); y. set. Left. Child(x. get. Right. Chil d()); x. set. Right. Child(y); x = y; y x A n Case 2: left. Rotate(x); n n y= x. get. Right. Child(); x. set. Right. Child(y. get. Left. Chil d()); y. set. Left. Child(x) x = y; x right. Rotate(y) C B Case 3: left. Rotate (y); right. Rotate (x); 27 -Apr-2009 n A left. Rotate(x) n y B C Case 4: right. Rotate(x); left. Rotate (y); Data Structure Review 11

AVL: Insert/Delete n Insertion is similar to a regular BST Insert: n Search for the position : n n n Keep going left (or right) in the tree until a null child is reached. n n 27 -Apr-2009 0 children: n n n replace it with the only child 2 children: n n replace it with null 1 child: n An inserted node is always a leaf. Search from inserted node to root looking for any node that violate the AVL property. n Search for the node. Remove it : n Rebalance the tree: n Deletion is similar to a regular BST Delete: Insert a new node in this position. n n n replace it with right-most node in the left subtree Rebalance the tree: n Use rotation to fix. Only require to find the first unbalanced node. Data Structure Review Search from inserted node to root for all node that violate the AVL property. n n Use rotation to fix. Require to work all the way back to the root. 12

Review: Red-Black Trees n Red-black trees extends BST data structure and include the following properties: n n n The root is always black. Every node is either red or black. Every leaf (NULL pointer) is black (every “real” node has 2 children). Both children of every red node are black (can’t have 2 consecutive reds on a path). Every simple path from node to descendent leaf contains the same number of black nodes. RB tree has height h 2 lg(n+1). n 27 -Apr-2009 So, operation is guaranteed to be the height h = O(lg n). Data Structure Review 13

RB Trees: Maintaining RB Tree Property n Tree rotation is the key operation for maintaining a RB tree property in O(lg n) time: y x A n n C B x right. Rotate(y) A left. Rotate(x) y B C Rotation preserves in-order key ordering Rotation takes O(1) time (just swaps pointers) 27 -Apr-2009 Data Structure Review 14

RB Trees: Insert/Delete n Insertion is similar to BST’s insert: n n n BST Insert. Color the new node red. Rebalance the tree: n n 2. 3. n Deletion is similar to BST’s delete: n n BST Delete; Rebalance the tree: n If parent is black, done. Otherwise: 1. 27 -Apr-2009 n n 1. Parent’s sibling is red. Parent’s sibling is black and new node is a right child. Parent’s sibling is black and new node is a left child. Repeat, moving up the tree until there are no violation. 2. 3. 4. n Data Structure Review If node is red, color black, done. Otherwise: Sibling has two black children. Sibling’s children are both black. Sibling's left child is red. sibling's right child is black, Sibling is black, sibling's right child is red. Repeat, moving up the tree until there are no violation. 15

Review: 2 -3 B-Trees n A B-tree extends tree data structure and has the following properties: n n n So, a 2 -3 B-tree is a B-tree of 3 order. n n The root is either a leaf or has between 2 and m children. Each internal node has between ceiling(m/2)-1 and m-1 keys. A leaf node has between 1 and m-1 keys. The tree is perfectly balanced. A node can have 2 or 3 children, which means that a node can have 1, 2 or 3 keys. R-B tree is a B-tree with degree 2. < x , y> <=x 27 -Apr-2009 Data Structure Review >x and <=y >y 16

2 -3: Insert/Delete n Insertion is similar to insert in a BST: n n n Searching for the item. If found, done. Otherwise, n n n Deletion is similar to delete in a BST. n n Stop at a 2 -node? n Upgrade the 2 -node to a 3 node. Stop at a 3 -node? n Replace the 3 -node by 2 2 nodes and push the middle value up to the parent node. Repeat recursively until you upgrade a 2 -node or create a new root. n n When is a new root created? 27 -Apr-2009 Start deletion at a leaf. Swap the value to be deleted with its immediate successor in the tree. Delete the value from the node. If the node still has a value, done. Data Structure Review We’ve changed a 3 -node into a 2 -node; Otherwise, n Find a value from sibling or parent. 17

Review: Hash Tables n Given n elements, each with a key and satellite data, we need to support: n n n insert (T, x), delete (T, x), and search(T, x), But, don’t care about sorting the elements. Suppose no two elements have the same key and the range of keys is 0…m-1, where m is not too large. n Set up an array T[0…m-1] in which. n n T[i] = x if x T and i=h(key(x)); T[i] = NULL otherwise. h() is called the hash function (or hashing) and T is called a direct-address table. Hash tables support insert, delete, and search in O(1) expected time. 27 -Apr-2009 Data Structure Review 18

Hash: Resolving Collisions n n Collision happens when two keys hash to the same memory location. Two ways to resolve collisions: n Open addressing: n n n Chaining: n n 27 -Apr-2009 To insert, if slot is full, try another slot, and another, until an open slot is found (probing). To search, follow same sequence of probes as would be used when inserting the element. To insert, keep linked list of elements in slots. Upon collision, just add new element to list. To search: search the linked list. Data Structure Review 19

Review: Graphs n A graph G = (V, E), where V = set of vertices, E = set of edges. n n n Dense graph: |E| |V|2 Sparse graph: |E| |V| Undirected graph: n n n Directed graph: n n Edge (u, v) = edge (v, u) No self-loops Edge (u, v) goes from vertex u to vertex v, notated u v A weighted graph associates weights with either the edges or the vertices. 27 -Apr-2009 Data Structure Review 21

Graphs: Adjacency Matrix n n Assume V = {1, 2, …, n}. An adjacency matrix represents the graph as a n x n matrix A: n A[i, j] = 1 if edge (i, j) E (or weight of edge) = 0 if edge (i, j) E 1 a d 2 b c 3 27 -Apr-2009 4 A 1 2 3 4 1 0 1 1 0 2 0 0 1 0 3 0 0 4 0 0 1 0 Data Structure Review 22

Graphs: Adjacency List n An adjacency list represents the graph as an array of linked list. n n 1 Example: n n n For each vertex v V, store a list of vertices adjacent to v. Adj[1] = {2, 3} Adj[2] = {3} Adj[3] = {} Adj[4] = {3} 2 Variation: can also keep a list of edges coming into vertex. 27 -Apr-2009 Data Structure Review 4 3 23

Graphs: Storage n Adjacency matrix takes O(V 2) storage. n n n Usually too much storage for large graphs. But can be very efficient for small graphs. Adjacency list takes O(V+E) storage: n n The degree of a vertex v = # incident edges. For directed graphs, # of items in adjacency lists is: n n For undirected graphs, # items in adjacency lists is: n n out-degree(v) = |E| takes (V + E) storage. degree(v) = 2 |E| (handshaking lemma) also takes (V + E) storage. Most large interesting graphs are sparse. n n 27 -Apr-2009 E. g. , planar graphs, in which no edges cross, have |E| = O(|V|) by Euler’s formula; So, the adjacency list is often a more appropriate representation. Data Structure Review 24

Review: Graph Searching n n n Given: a graph G = (V, E), directed or undirected. Goal: systematically explore every vertex and every edge. General idea: build a tree on the graph. n n n 27 -Apr-2009 Pick a vertex as the root, Choose certain edges to produce a tree. Note: might also build a forest if graph is not connected. Data Structure Review 25

Breadth-First Search n General idea: n n Expand frontier of explored vertices across the breadth of the frontier. Pick a source vertex to be the root. Find (“discover”) its children, then their children, etc. Associate vertex “colours”: n White vertices have not been discovered. n n Grey vertices are discovered but not fully explored. n n They may be adjacent to white vertices. Black vertices are discovered and fully explored. n n All vertices start out white. They are adjacent only to black and grey vertices. Explore vertices by scanning the FIFO queue of grey vertices. 27 -Apr-2009 Data Structure Review 26

BFS and Shortest-path n n BFS can thought of as being like Dijkstra’s for shortest-path except every edge has the same weight. BFS calculates the shortest-path distance to the source node. n n n Shortest-path distance (s, v) = minimum number of edges from s to v, or if v not reachable from s. Proof should be in the book. BFS builds breadth-first tree, in which paths to root represent shortest paths in G. n 27 -Apr-2009 Thus can use BFS to calculate shortest path from one vertex to another in O(V+E) time. Data Structure Review 27

Depth-First Search n General idea: n n Like BFS, associate vertex “colours”: n n Explore “deeper” in the graph whenever possible. Edges are explored out of the most recently discovered vertex v that still has unexplored edges. When all of v’s edges have been explored, backtrack to the vertex from which v was discovered. Vertices initially white. Then coloured grey when discovered. Then coloured black when finished. Explore vertices by scanning the stack of grey vertices 27 -Apr-2009 Data Structure Review 28

DFS And Cycles n An undirected graph is acyclic iff a DFS yields no back edges n n If acyclic, no back edges (because a back edge implies a cycle). If no back edges, acyclic. n No back edges implies only tree edges (Why? ) n n n Only tree edges implies we have a tree or a forest, which by definition is acyclic. Thus, can run DFS to find whether a graph has a cycle. We can actually determine if cycles exist in O(V) time: n n 27 -Apr-2009 In an undirected acyclic forest, |E| |V| - 1. So count the edges: if ever see |V| distinct edges, must have seen a back edge along the way. Data Structure Review 29

Remarks (1) n Clearly data structures and algorithms are closely related: n n However, consideration of many factors are required to produce a good implementation: n n n Selecting the most efficient data structure and algorithm will almost always be the best way to proceed. The obvious solution isn’t always the best. Sometimes it makes sense to have multiple data structures, each with different properties, to represent a single object. Factors to be considered are: n n 27 -Apr-2009 The memory footprint implied by a given representation. The cost of operations in that representation. The cost of converting to another representation. The amount of computation expected using a given representation. Data Structure Review 30

Remarks (2) n When it comes to the implementation of an algorithm, the main point is that constant factors matter. n n Often, require to restructure a program, not functionally but behaviourally, to get better performance. n n Mapping algorithms and data structures in a way that matches the architecture characteristics is VERY important ! However, restructuring code, can be a bit more involved than just performing optimisations. So, the bottom line is to think about trade-offs that could change the quality of an implementation. n n 27 -Apr-2009 Direct, obvious algorithm translations don’t always mean good performance; Best performance comes from considering the many aspects of execution, e. g. , memory access, processor characteristics, language overheads. Data Structure Review 31

Good Luck! 27 -Apr-2009 Data Structure Review 32