Hashing B Trees and Red Black Trees using
Hashing, B - Trees and Red – Black Trees using Parallel Algorithms & Sequential Algorithms By Yazeed K. Almarshoud
Road map Introduction. n Definitions. n ¨ Hashing. ¨ B Trees ¨ Red Black Trees. Sequential algorithms n Parallel algorithms n 6/4/2021 2
Introduction n In this presentation I ‘m glad to present to you the importance of parallel computations and how it is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones which are then solved concurrently 6/4/2021 3
Hashing
Definitions n Hashing: A hash function is any well – defined procedure or mathematical function which converts a large, possibly variablesized amount of data into a small datum, usually a single integer that may serve as an index into an array. n i. e. keys made up for alphabetical characters could be replaced by their ASCII equivalents. n Two standards hashing techniques: ¨ ¨ 6/4/2021 Division methods Multiplication method. 5
Symbol – table problem 6/4/2021 6
Hash functions 6/4/2021 7
Choosing a hash function The assumption of simple uniform hashing is hard to guarantee, but several common techniques tend to work well in practice as long as their deficiencies can be avoided. n Desirata: § A good hash function should distribute the keys uniformly into the slots of the table. Regularity in the key distribution should not affect this uniformity. § 6/4/2021 8
Division method n Assume all keys are integers, and define h(k) = k mod m. n Deficiency: Don’t pick an m that has a small divisor d. A preponderance of keys that are congruent modulo d can adversely affect uniformity. Extreme deficiency: If m = 2 r, then the hash doesn’t even depend on all the bits of k: If k = 10110001110110102 and r = 6, then h(k) = 0110102. n 6/4/2021 9
Division method (continued) h(k) = k mod m. Pick m to be a prime not too close to a power of 2 or 10 and not otherwise used prominently in the computing environment. n Annoyance: Sometimes, making the table size a prime is inconvenient. But, this method is popular, although the next method we’ll see is usually superior. n 6/4/2021 10
Multiplication method Assume that all keys are integers, m = 2 r, and our computer has w-bit words. Define h(k) = (A·k mod 2 w) rsh (w – r), where rsh is the “bit-wise right-shift” operator and A is an odd integer in the range 2 w– 1 < A < 2 w. n Don’t pick A too close to 2 w. n Multiplication modulo 2 w is fast. n The rsh operator is fast. n 6/4/2021 11
Multiplication method example 6/4/2021 12
Resolving collisions by chaining 6/4/2021 13
Analysis of chaining 6/4/2021 14
Search cost 6/4/2021 15
Resolving collisions by open addressing n n n No storage is used outside of the hash table itself. . The hash function depends on both the key and probe number: h : U X {0, 1, …, m– 1}. E. g. h(k) = (k+i) mod m ; h(k) = (k+i 2 ) mod m Inserting a key k: ¨ ¨ ¨ n Finding a key k: ¨ ¨ ¨ n we check T[h(k, 0)]. If empty we insert k, there. Otherwise, we check T[h(k, 1)]. If empty we insert k, there. Otherwise, … otherwise etc for h(k, 2), h(k, 1), …, h(k, m– 1). we check if T[h(k, 0)] is empty, and if =k. If not we check if T[h(k, 1)] is empty, and if =k. If not otherwise etc for h(k, 2), h(k, 1), …, h(k, m– 1). Deleting a key k ¨ 6/4/2021 Find it are replace with a dummy (why) 16
Example of open addressing 6/4/2021 17
Example of open addressing 6/4/2021 18
Example of open addressing 6/4/2021 19
Example of open addressing 6/4/2021 20
Probing strategies n n Linear probing: Given an ordinary hash function h ‘(k), linear probing uses the hash function h(k, i) = (h’(k) + i) mod m. n This method, though simple, suffers from primary clustering, where long runs of occupied slots build up, increasing the average search time. Moreover, the long runs of occupied slots tend to get longer. 6/4/2021 21
Red-Black Trees v 6 8 3 4 z
Roadmap ¨ Definition ¨ Height ¨ Insertion restructuring n recoloring n ¨ Deletion restructuring n recoloring n adjustment n 6/4/2021 23
Red-Black Tree n A red-black tree can also be defined as a binary search tree that satisfies the following properties: Root Property: the root is black ¨ External Property: every leaf is black ¨ Internal Property: the children of a red node are black ¨ Depth Property: all the leaves have the same black depth ¨ 9 4 2 15 6 12 21 7 6/4/2021 24
Height of a Red-Black Tree n n n 6/4/2021 Theorem: A red-black tree storing n items has height O(log n) The search algorithm for a red-black search tree is the same as that for a binary search tree By the above theorem, searching in a red-black tree takes O(log n) time 25
Insertion n To perform operation insert. Item(k, o), we execute the insertion algorithm for binary search trees and color red the newly inserted node z unless it is the root We preserve the root, external, and depth properties ¨ If the parent v of z is black, we also preserve the internal property and we are done ¨ Else (v is red ) we have a double red (i. e. , a violation of the internal property), which requires a reorganization of the tree ¨ n Example where the insertion of 4 causes a double red: v 6/4/2021 6 3 8 z v 6 8 3 4 z 26
Remedying a Double Red n Consider a double red with child z and parent v, and let w be the sibling of v Case 1: w is black Case 2: w is red The double red is an incorrect replacement of a 4 -node ¨ Restructuring: we change the 4 -node replacement ¨ w 4 2 z 7 6 4 6 7 v The double red corresponds to an overflow ¨ Recoloring: we perform the equivalent of a split ¨ w 4 2 z 7 v 6 2 4 6 7 . . 2. . 6/4/2021 27
n Local invariants example: ¨ It involves only the fields of an object and the fields of its tree-children ¨ We specify local invariants using the rep. Ok. Local method. 6/4/2021 28
Restructuring n n n A restructuring remedies a child-parent double red when the parent red node has a black sibling It is equivalent to restoring the correct replacement of a 4 -node The internal property is restored and the other properties are preserved z 6 4 v v w 7 7 2 4 z w 2 6 4 6 7. . 2. . 6/4/2021 4 6 7. . 29
Restructuring (cont. ) n There are four restructuring configurations depending on whether the double red nodes are left or right children 6 2 6 4 6 2 4 4 2 2 4 6 4 2 6/4/2021 6 30
Recoloring n n n A recoloring remedies a child-parent double red when the parent red node has a red sibling The parent v and its sibling w become black and the grandparent u becomes red, unless it is the root The double red violation may propagate to the grandparent u w 4 2 z 7 v w 4 2 6 7 6 … 4 … 2 4 6 7 2 6/4/2021 z v 6 7 31
Analysis of Insertion Algorithm insert. Item(k, o) n 1. We search for key k to locate the insertion node z n 2. We add the new item (k, o) at node z and color z red n 3. while double. Red(z) if is. Black(sibling(parent(z))) z restructure(z) return else { sibling(parent(z) is red } z recolor(z) 6/4/2021 n Recall that a red-black tree has O(log n) height Step 1 takes O(log n) time because we visit O(log n) nodes Step 2 takes O(1) time Step 3 takes O(log n) time because we perform O(log n) recolorings, each taking O(1) time, and ¨ at most one restructuring taking O(1) time ¨ n Thus, an insertion in a redblack tree takes O(log n) time 32
Deletion n n To perform operation remove(k), we first execute the deletion algorithm for binary search trees Let v be the internal node removed, w the external node removed, and r the sibling of w If either v of r was red, we color r black and we are done ¨ Else (v and r were both black) we color r double black, which is a violation of the internal property requiring a reorganization of the tree ¨ n Example where the deletion of 8 causes a double black: 6 8 3 4 6/4/2021 r 6 v w 3 r 4 33
Remedying a Double Black n The algorithm for remedying a double black node w with sibling y considers three cases Case 1: y is black and has a red child ¨ We perform a restructuring, equivalent to a transfer , and we are done Case 2: y is black and its children are both black ¨ We perform a recoloring, equivalent to a fusion, which may propagate up the double black violation Case 3: y is red ¨ n 6/4/2021 We perform an adjustment, equivalent to choosing a different representation of a 3 -node, after which either Case 1 or Case 2 applies Deletion in a red-black tree takes O(log n) time 34
Red-Black Tree Reorganization Insertion remedy double red Red-black tree action (2, 4) tree action result restructuring change of 4 -node representation double red removed recoloring split double red removed or propagated up Deletion remedy double black Red-black tree action (2, 4) tree action result restructuring transfer double black removed recoloring fusion double black removed or propagated up adjustment change of 3 -node representation restructuring or recoloring follows 6/4/2021 35
Binary Trees 6 3 8
Binary Trees n A tree in which no node can have more than two children n The depth of an “average” binary tree is considerably smaller than N, even though in the worst case, the depth can be as large as N – 1. 6/4/2021 37
Example: Expression Trees n n n Leaves are operands (constants or variables) The other nodes (internal nodes) contain operators Will not be a binary tree if some operators are not binary 6/4/2021 38
Binary Trees n Possible operations on the Binary Tree ADT ¨ parent ¨ left_child, right_child ¨ sibling ¨ root, etc n Implementation ¨ Because a binary tree has at most two children, we can keep direct pointers to them 6/4/2021 39
Compare: Implementation of a general tree 6/4/2021 40
Binary Search Trees n n Stores keys in the nodes in a way so that searching, insertion and deletion can be done efficiently. Binary search tree property ¨ 6/4/2021 For every node X, all the keys in its left subtree are smaller than the key value in X, and all the keys in its right subtree are larger than the key value in X 41
Binary Search Trees A binary search tree 6/4/2021 Not a binary search tree 42
Binary search trees Two binary search trees representing the same set: n Average depth of a node is O(log N); maximum depth of a node is O(N) 6/4/2021 43
Implementation 6/4/2021 44
Searching BST n n n If we are searching for 15, then we are done. If we are searching for a key < 15, then we should search in the left subtree. If we are searching for a key > 15, then we should search in the right subtree. 6/4/2021 45
6/4/2021 46
Searching (Find) n Find X: return a pointer to the node that has key X, or NULL if there is no such node n Time complexity ¨ 6/4/2021 O(height of the tree) 47
Inorder traversal of BST n Print out all the keys in sorted order 6/4/2021 Inorder: 2, 3, 4, 6, 7, 9, 13, 15, 17, 18, 20 48
Find. Min/ Find. Max n n Return the node containing the smallest element in the tree Start at the root and go left as long as there is a left child. The stopping point is the smallest element Similarly for find. Max Time complexity = O(height of the tree) 6/4/2021 49
Insert n n Proceed down the tree as you would with a find If X is found, do nothing (or update something) Otherwise, insert X at the last spot on the path traversed Time complexity = O(height of the tree) 6/4/2021 50
Delete n When we delete a node, we need to consider how we take care of the children of the deleted node. ¨ This has to be done such that the property of the search tree is maintained. 6/4/2021 51
Delete Three cases: (1) the node is a leaf ¨ Delete it immediately (2) the node has one child ¨ 6/4/2021 Adjust a pointer from the parent to bypass that node 52
Delete (3) the node has 2 children ¨ replace the key of that node with the minimum element at the right subtree ¨ delete the minimum element n n Has either no child or only right child because if it has a left child, that left child would be smaller and would have been chosen. So invoke case 1 or 2. Time complexity = O(height of the tree) 6/4/2021 53
Sequential algorithms
Sequential algorithms n Is the search space a tree or a graph? n The space of a 0/1 integer program is a tree, while that of an 8 -puzzle is a graph. n This has important implications for search since unfolding a graph into a tree can have significant overheads. 6/4/2021 55
Sequential algorithms Two examples of unfolding a graph into a tree. 6/4/2021 56
Best-First Search (BFS) Algorithms n BFS algorithms use a heuristic to guide search. n The core data structure is a list, called Open list, that stores unexplored nodes sorted on their heuristic estimates. n The best node is selected from the list, expanded, and its off-spring are inserted at the right position. n If the heuristic is admissible, the BFS finds the optimal solution. 6/4/2021 57
Best-First Search (BFS) Algorithms n BFS of graphs must be slightly modified to account for multiple paths to the same node. n A closed list stores all the nodes that have been previously seen. n If a newly expanded node exists in the open or closed lists with better heuristic value, the node is not inserted into the open list. 6/4/2021 58
Best-First Search: Example Applying best-first search to the 8 -puzzle: (a) initial configuration; (b) final configuration; and (c) states resulting from the first four steps of best-first search. Each state is labeled with its -value (that is, the Manhattan distance from the state to the final state). 59 6/4/2021
Search Overhead Factor n The amount of work done by serial and parallel formulations of search algorithms is often different. n Let W be serial work and WP be parallel work. Search overhead factor s is defined as WP/W. n Upper bound on speedup is p×(W/WP). 6/4/2021 60
Parallel algorithms
Parallel algorithms n n n How is the search space partitioned across processors? Different subtrees can be searched concurrently. However, subtrees can be very different in size. It is difficult to estimate the size of a subtree rooted at a node. Dynamic load balancing is required. 6/4/2021 62
Parallel algorithms n When a processor runs out of work, it gets more work from another processor. n This is done using work requests and responses in message passing machines and locking and extracting work in shared address space machines. n On reaching final state at a processor, all processors terminate. n Unexplored states can be conveniently stored as local stacks at processors. n The entire space is assigned to one processor to begin with. 6/4/2021 63
Parallel Best-First Search n The core data structure is the Open list (typically implemented as a priority queue). n Each processor locks this queue, extracts the best node, unlocks it. n Successors of the node are generated, their heuristic functions estimated, and the nodes inserted into the open list as necessary after appropriate locking. n Termination signaled when we find a solution whose cost is better than the best heuristic value in the open list. n Since we expand more than one node at a time, we may expand nodes that would not be expanded by a sequential algorithm. 6/4/2021 64
Parallel Best-First Search A general schematic for parallel best-first search using a centralized strategy. The locking operation is used here to serialize queue access by various processors. 6/4/2021 65
Parallel Best-First Search n The open list is a point of contention. n Let texp be the average time to expand a single node, and taccess be the average time to access the open list for a single-node expansion. n If there are n nodes to be expanded by both the sequential and parallel formulations (assuming that they do an equal amount of work), then the sequential run time is given by n (taccess + texp). n The parallel run time will be at least n taccess. n Upper bound on the speedup is (taccess+ texp)/taccess 6/4/2021 66
Parallel Best-First Search n Avoid contention by having multiple open lists. n Initially, the search space is statically divided across these open lists. n Processors concurrently operate on these open lists. n Since the heuristic values of nodes in these lists may diverge significantly, we must periodically balance the quality of nodes in each list. n A number of balancing strategies based on ring, blackboard, or random communications are possible. 6/4/2021 67
Parallel Best-First Search A message-passing implementation of parallel best-first search using the ring communication strategy. 6/4/2021 68
Parallel Best-First Search An implementation of parallel best-first search using the blackboard communication strategy. 6/4/2021 69
Parallel Best-First Graph Search n Graph search involves a closed list, where the major operation is a lookup (on a key corresponding to the state). n The classic data structure is a hash. n Hashing can be parallelized by using two functions - the first one hashes each node to a processor, and the second one hashes within the processor. n This strategy can be combined with the idea of multiple open lists. n If a node does not exist in a closed list, it is inserted into the open list at the target of the first hash function. n In addition to facilitating lookup, randomization also equalizes quality of nodes in various open lists. 6/4/2021 70
Speedup Anomalies in Parallel Search n Since the search space explored by processors is determined dynamically at runtime, the actual work might vary significantly. n Executions yielding speedups greater than p by using p processors are referred to as acceleration anomalies. Speedups of less than p using p processors are called deceleration anomalies. n Speedup anomalies also manifest themselves in best-first search algorithms. n If the heuristic function is good, the work done in parallel best-first search is typically more than that in its serial counterpart. 6/4/2021 71
Speedup Anomalies in Parallel Search The difference in number of nodes searched by sequential and parallel formulations of DFS. For this example, parallel DFS reaches a goal node after searching fewer nodes than sequential DFS. 6/4/2021 72
Speedup Anomalies in Parallel Search A parallel DFS formulation that searches more nodes than its sequential counterpart 6/4/2021 73
To here, The Presentation ends By Yazeed K. Almarshoud CS 6260
Presentation Question: What are the two methods of Resolving collusions in addressing hashing functions mentioned in the presentation? In Parallel systems, when a processor runs out of work, it gets more work from another processor (How? ). §Part One: § The two methods are: § Resolving §Part collusion by chaining. collusion by open addressing. Two: § This is done using work requests and responses in message passing machines and locking and extracting work in shared address space machines.
- Slides: 75