Data Structures Hash Tables Hashing Tables l Motivation

Data Structures Hash Tables

Hashing Tables l Motivation: symbol tables n A compiler uses a symbol table to relate symbols to associated data u Symbols: variable names, procedure names, etc. u Associated data: memory location, call graph, etc. n n For a symbol table (also called a dictionary), we care about search, insertion, and deletion We typically don’t care about sorted order

Hash Tables l More formally: n Given a table T and a record x, with key (= symbol) and satellite data, we need to support: Insert (T, x) u Delete (T, x) u Search(T, x) u n l We want these to be fast, but don’t care about sorting the records The structure we will use is a hash table n Supports all the above in O(1) expected time!

Hashing: Keys In the following discussions we will consider all keys to be (possibly large) natural numbers l How can we convert floats to natural numbers for hashing purposes? l How can we convert ASCII strings to natural numbers for hashing purposes? l

Hashing a String Let Ai represent the ascii code for a string character. Since ascii codes go from 0 to 127 we can perform the following calculation. A 3 X 3 + A 2 X 2 + A 1 X 1 +A 0 X 0 = (((A 3)X + A 2)X + A 1)X + A 0 What is this called? overflow. Solution: Note that if X = 128 we can easily public static int hash(String key, int table. Size) { int hash. Val = 0; for(int i=0; i < key. length(); i++) hash. Val = ( hash. Val * 128 + key. char. At(i))%table. Size; return hash. Val; }

A better version Hash functions should be fast and distribute the keys equitably! The 37 slows down the shifting of lower keys and we allow overflow to occur(sort of auto mod-ing) which can result in a neg. public static int hash(String key, int table. Size) { int hash. Val = 0; for(int i=0; i < key. length(); i++) hash. Val = hash. Val * 37 + key. char. At(i); hash. Val %= table. Size; if( hash. Val < 0)hash. Val += table. Size; // fix the sign return hash. Val; }

Direct Addressing l Suppose: n n l The range of keys is 0. . m-1 Keys are distinct The idea: n Set up an array T[0. . m-1] in which u T[i] =x u T[i] = NULL n if x T and key[x] = i otherwise This is called a direct-address table u Operations take O(1) time! u So what’s the problem?

The Problem With Direct Addressing Direct addressing works well when the range m of keys is relatively small l But what if the keys are 32 -bit integers? l n n Problem 1: direct-address table will have 232 entries, more than 4 billion Problem 2: even if memory is not an issue, the time to initialize the elements to NULL may be Solution: map keys to smaller range 0. . m-1 l This mapping is called a hash function l

Hash Functions l Next problem: collision T U (universe of keys) h(k 1) h(k 4) k 1 k 4 K (actual keys) k 2 0 k 5 h(k 2) = h(k 5) k 3 h(k 3) m-1

Resolving Collisions How can we solve the problem of collisions? l Solution 1: chaining l Solution 2: open addressing l

Open Addressing l Basic idea: n n To insert: if slot is full, try another slot, …, until an open slot is found (probing) To search, follow same sequence of probes as would be used when inserting the element If reach element with correct key, return it u If reach a NULL pointer, element is not in table u l Good for fixed sets (adding but no deletion) n l Example: spell checking Table needn’t be much bigger than n

Open Addressing(Linear Probing) Let H(k, p) denote the pth position tried for key K where p=0, 1, . . . Thus H(k, 0)= h(k) where h is the basic hash function. H(k, p) represents the probe sequence we use to find the key. If key K hashes to index i, and it is full, then try i+1, i+2, until an empty slot is found. The Linear Probing sequence is therefore shown below. H(K, 0) = h(K) H(K, p+1) = (H(K, p) + 1) mod m

Open Addressing(Double Hashing) Here we use a second hash function h 2 to determine the probe sequence. Note that linear probing is just h 2(K)=1 H(K, 0) = h(K) H(K, p+1) = (H(K, p) + h 2(K)) mod m To ensure that the probe sequence visits all positions in the table h 2(K) must be greater than zero and relatively prime to m for any K. Hence we usually let m be prime.

Mathematical Defn’s Let us count as one probe each access to the data structure and n the size of the dictionary to be stored and m the size of the table. Let =n/m be called the load factor. From a statistical stand point let S( ) be the expected number of probes to perform a Look. Up on a key that is actually in the hash table, and U( ) be the expected number of probes in an unsuccessful Look. Up.

Performance of Linear Probing The names were inserted in alphabetical order. The third column shows the number of probes required to find a key in the table; this is the same as the number of slots that were inspected when it was inserted. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 # probes S. Adams 1 J. Adams W. Floyd T. Heyward 1 2 1 J. Hancock 1 C. Braxton J. Hart J. Hewes 1 1 3 C. Carroll A. Clark R. Ellery B. Franklin W. Hooper 1 1 2 1 5 primary clustering S = 21/13=1. 615 U = 47/18=2. 6 load is 13/18=. 72 key

Linear Probing Knuth analyzed sequential probing and obtained the following. The small extra effort required to implement double hashing certainly pays off it seems. =. 9 Sn Un Linear Double 5. 5 50. 5 2. 56 10. 0

Performance of Double Hashing Assumption: each probe inot the hash table is independent and has a probability of hitting an occupied position exactly equal to the load factor. Let i = i/m for every i n. This is then the probability of a collision on any probe after i keys have been inserted. The expected number of probes in an unsuccessful search when n 1 items have been inserted is Un-1 1(1 - n-1)+2 n-1(1 - n-1)+3 n-12 (1 - n-1)+. . . = 1 + n-12 +. . . = 1/(1 - n-1 ) Prob. that third probe is empty and first 2 filled

Successful Searching(Double) The # of probes in a successful search is the average of the number of probes it took to insert each of the n items. The expected number of probes to insert the ith item is the expected # of probes in an unsuccessful search when i-1 items have been inserted.

Successful Search Continued Where So

Observation(Double Hashing) When n = 20/31 we have Sn 1. 606 Even when the table is 90% full Sn is 2. 56 although Un 1/1 -. 9 = 10. 0

Chaining l Chaining puts elements that hash to the same slot in a linked list: T U (universe of keys) k 6 k 2 k 5 k 8 k 1 k 4 —— k 5 k 2 —— —— —— k 1 k 4 K (actual k 7 keys) —— k 3 —— k 8 —— k 6 —— k 7 ——

Chaining l How do we insert an element? U (universe of keys) k 6 k 2 k 5 k 8 k 1 k 4 —— k 5 k 2 —— —— —— k 1 k 4 K (actual k 7 keys) T —— k 3 —— k 8 —— k 6 —— k 7 ——

Chaining l How do we delete an element? U (universe of keys) k 6 k 2 k 5 k 8 k 1 k 4 —— k 5 k 2 —— —— —— k 1 k 4 K (actual k 7 keys) T —— k 3 —— k 8 —— k 6 —— k 7 ——

Chaining l How do we search for a element with a given key? T U (universe of keys) k 6 k 2 k 5 k 8 k 1 k 4 —— k 5 k 2 —— —— —— k 1 k 4 K (actual k 7 keys) —— k 3 —— k 8 —— k 6 —— k 7 ——

Analysis of Chaining Assume simple uniform hashing: each key in table is equally likely to be hashed to any slot l Given n keys and m slots in the table: the load factor = n/m = average # keys per slot l What will be the average cost of an unsuccessful search for a key? l

Analysis of Chaining Assume simple uniform hashing: each key in table is equally likely to be hashed to any slot l Given n keys and m slots in the table, the load factor = n/m = average # keys per slot l What will be the average cost of an unsuccessful search for a key? A: O(1+ ) l

Analysis of Chaining Assume simple uniform hashing: each key in table is equally likely to be hashed to any slot l Given n keys and m slots in the table, the load factor = n/m = average # keys per slot l What will be the average cost of an unsuccessful search for a key? A: O(1+ ) l What will be the average cost of a successful search? l

Analysis of Chaining Assume simple uniform hashing: each key in table is equally likely to be hashed to any slot l Given n keys and m slots in the table, the load factor = n/m = average # keys per slot l What will be the average cost of an unsuccessful search for a key? A: O(1+ ) l What will be the average cost of a successful search? A: O(1 + /2) = O(1 + ) l

Analysis of Chaining Continued So the cost of searching = O(1 + ) l If the number of keys n is proportional to the number of slots in the table, what is ? l A: = O(1) l n In other words, we can make the expected cost of searching constant if we make constant

Choosing A Hash Function l Clearly choosing the hash function well is crucial n n l What will a worst-case hash function do? What will be the time to search in this case? What are desirable features of the hash function? n n Should distribute keys uniformly into slots Should not depend on patterns in the data

Hash Functions: The Division Method l h(k) = k mod m n In words: hash k into a table with m slots using the slot given by the remainder of k divided by m What happens to elements with adjacent values of k? l What happens if m is a power of 2 (say 2 P)? l What if m is a power of 10? l Upshot: pick table size m = prime number not too close to a power of 2 (or 10) l

Hash Functions: The Multiplication Method For a constant A, 0 < A < 1: l h(k) = m (k. A - k. A ) l What does this term represent?

Hash Functions: The Multiplication Method For a constant A, 0 < A < 1: l h(k) = m (k. A - k. A ) l Fractional part of k. A Choose m = 2 P l Choose A not too close to 0 or 1 l Knuth: Good choice for A = ( 5 - 1)/2 l

Hash Functions: Worst Case Scenario l Scenario: n n n You are given an assignment to implement hashing You will self-grade in pairs, testing and grading your partner’s implementation In a blatant violation of the honor code, your partner: u Analyzes your hash function u Picks a sequence of “worst-case” keys, causing your implementation to take O(n) time to search l What’s an honest CS student to do?

Hash Functions: Universal Hashing As before, when attempting to foil an malicious adversary: randomize the algorithm l Universal hashing: pick a hash function randomly in a way that is independent of the keys that are actually going to be stored l n Guarantees good performance on average, no matter what keys adversary chooses

Universal Hashing l Let be a (finite) collection of hash functions n n l …that map a given universe U of keys… …into the range {0, 1, …, m - 1}. is said to be universal if: n n for each pair of distinct keys x, y U, the number of hash functions h for which h(x) = h(y) is | |/m In other words: a random hash function from , the chance of a collision between x and y (x y) is exactly 1/m u With

Universal Hashing l Theorem 12. 3: n n Choose h from a universal family of hash functions Hash n keys into a table of m slots, n m Then the expected number of collisions involving a particular key x is less than 1 Proof: For each pair of keys y, z, let cyx = 1 if y and z collide, 0 otherwise u E[cyz] = 1/m (by definition) u Let Cx be total number of collisions involving key x u u u Since n m, we have E[Cx] < 1

A Universal Hash Function Choose table size m to be prime l Decompose key x into r+1 bytes, so that x = {x 0, x 1, …, xr} l n Only requirement is that max value of byte < m Let a = {a 0, a 1, …, ar} denote a sequence of r+1 elements chosen randomly from {0, 1, …, m - 1} Define corresponding hash function ha : n With this definition, has mr+1 members n n

A Universal Hash Function l is a universal collection of hash functions (Theorem 12. 4) l How to use: n n n Pick r based on m and the range of keys in U Pick a hash function by (randomly) picking the a’s Use that hash function on all keys

The End