Randomized Algorithms Lecturer Moni Naor Cuckoo Hashing The

Randomized Algorithms Lecturer: Moni Naor Cuckoo Hashing

The Setting • Dynamic dictionary: – Lookups, insertions and deletions – Dynamic vs. static • Performance: – – Lookup time Update time Memory utilization Related problems: § Approximate set membership § Retrieval 2

Major Problem in Hashing Based Schemes: Collisions • Hash Tables – Hash Function h Table T – Direct access to memory § Element x should be in T[h(x)]. . . Unless |T| is |U| U T – Collision two different elements with the same value under h

Dealing with Collisions • Lots of methods • Common one: : Linear Probing – Proposed by Amdahl – Analyzed by Donald Knuth 1963 • “Birth” of analysis of algorithms – Probabilistic analysis 4

Linear Probing To insert x : try location h(x) in table T – If occupied try h(x)+1 – and if occupied try h(x)+2 – Until we find a vacant place • Looking for x: • Similar to insert: search until finding x or a vacant place. 5

Linear Probing: situation after a while 6

Cuckoo Hashing: Basics • Introduced by Pagh and Rodler (2001) • Extremely simple: – 2 tables: T 1 and T 2 § Each of size r = (1+ε)n h 1(x) – 2 hash functions: h 1 and h 2 t y b c x a – Check in T 1 and T 2 T 1 Where is x? h 2(x) . . . • Lookup: d z T 2 7

Cuckoo Hashing: Insertion Algorithm To element • insert To insert x: x, call Insert(x, 1) – Puti): in first location and displace residing Insert(x, if needed 1. Put xelement into location hi(x) in Ti 2. If T–i[h was empty: return in its other location Put displaced element i(x)] 3. If – Ti[h y: do Insert(y, 3–i) i(x)] contained Until finding a element free spot d Example: h 1(e) = h 1(a) c x a T 1 e h 2(y) = h 2(a) . . . h 1(y) t y b z T 2 8

Cuckoo Hashing: Insertion What happens if we are not successful? • Unsuccessful stems from two reasons – Not enough space § The path goes into loops – Too long chains • Can detect both by a time limit. 9

Cuckoo Hashing: Deletion • Extremely simple: – 2 tables: T 1 and T 2 § Each of size r = (1+ε)n – 2 hash functions: h 1 and h 2 h 1(x) • As in Lookup: t y b c x a T 1 Where is x? h 2(x) . . . – Check in T 1 and T 2 – Remove wherever found d z T 2 10

The Cuckoo Graph Set S ⊂ U containing n elements Insertion h 1, h 2 : U {0, . . . , r-1} algorithm Bipartite graph with |L|=|R|=r Edge (h 1(x), h 2(x)) for every x∈S achieves this Facts: S is successfully stored Every connected component in the cuckoo graph has at most one cycle occupied vacant Expected insertion time: O(1) 11

Cuckoo Hashing: Properties • Many attractive properties: – Lookup and deletion in 2 accesses in the worst case § May be done in parallel – Insertion takes amortized constant time – Reasonable memory utilization – No dynamic memory allocation • Lots of Generalizations • Lots of applications to related problems • Growing evidence of practicality on current architectures 12

Animation of Insertions T 1 Input Queue With queue for deamortization T 2 0 1 2 3 4 5 6 7 8 9 10 11 h 11( h 22( h 1( h 2( 8 6 2 ) = 410 ) = 10 4168 )=6 )=8 13

More than One Cycle • With probability Θ(1/n) Cuckoo Hashing fails – Recall: connected component with more than one cycle – Standard solution: rehashing § Meaning: insertion worst case may be O(n) • [KMW 08] suggested stash Kirsch, Mitzenmacher, – Saves rehashing Wieder § Actually reduces insertion worst case to O(log n) – Constant-sized stash is sufficient whp Prob of more than k: 1/nk 14

The Stash • Because of stash, need also Cycle Detection Mechanism: – Can accommodate log n elements – Supporting O(1) lookups and resets – Detecting second cycle as it happens. . . Queue Cuckoo graph 15

Towards Limited Independence • So far: truly random functions • Really want: – Succinct representation: o(n) – Fast evaluation: O(1) – Close-to-random behavior • Approach: use k-wise independence for smallest k possible – Good news: many such efficient constructions [Sie 89, ÖP 03, DW 03, . . . ] Dietzfelbinger, • But: Siegel doesÖstlin, k-wise independence suffice for k < n? Pagh Woelfel – I. e. , does analysis still hold? 16

k-wise Independent Hash Functions • Family H of hash functions {hi}hi ∈H • i hi : U [0, . . . , m-1] Definition: A family H of hash functions is k-wise independent if for any distinct x 1, . . . , xk∈U and for any y 1, . . . , yk∈[0, . . . , m-1]: k Pr [h(x )=y. . . h(x )=y ] = 1/m 1 1^ 2 2^ k k ^ h∈H 17

Boolean Circuits and Randomness • Boolean circuit: – DAG – size – fan-in – n inputs, m outputs – depth • AC 0 circuit: poly size, constant depth, unbounded fan-in • Until recently: – [LN 90] conjecture: any function computable by depth d, size s circuit is fooled by (log s)d– 1 -wise independence Linial, Nisan Fooled: probability of outputting 1 close on both distributions 18

Braverman’s Result • Braverman (2009): Theorem: Let be 0 circuit is offooled depth by d and size m. Any. CAC polylog(n)-wise distribution. Let μ be r-independent distribution for 2) d O(d r ≥ c · (log m) Special case of DNF formulas Then C cannot μ from uniform distribution. provedtell by Bazzi andthe Razborov – Note: for m = n. O(log n), r is polylog(m) 19

Back to Our Story: Limited Independence • Recall: the randomness assumption in analysis used for two events: – Event 1: sum of sizes of connected components is larger than a threshold – Event 2: the number of stashed elements is larger than a threshold • Each of these events: can be recognized by depth 3 circuit of size n. O(log n) h (x ) • The techniqueh (xis) quite general: h (x ) 1 1 . . . 2 1 . . . 1 2 . . . 2 2 . . . 1 k . . . 2 k . . . Kirsch, Mitzenmacher, Wieder – May be useful in similar scenarios – For example, implies a similar result for [KMW 08] 0/1 20

References (1) [Bra 09] Braverman – Poly-logarithmic independence fools AC 0 circuits, CCC 2009. [BT 93] Briggs, Torczon – An efficient representation for sparse sets, LOPLAS 1993 [DW 07] Dietzfelbinger, Weidling – Balanced allocation and dictionaries with tightly packed constant size bins, Theor. Comp. Sc. 2007 [FPS+05] Fotakis, Pagh, Sanders, Spirakis – Space efficient hash tables with worst case constant access time, Theor. Comp. Sys. , 2005 [KM 07] Kirsch, Mitzenmacher – Using a queue to de-amortize cuckoo hashing in hardware, ALLERTON 2007 [KMW 08] Kirsch, Mitzenmacher, Wieder – More robust hashing: Cuckoo hashing with a stash, ESA 2008 21

References (2) [PR 04] [Pan 05] [Ros 06] [Sun 93] [ZHB 06] Pagh, Rodler – Cuckoo hashing, J. Algs 2004 Panigrahy – Efficient hashing with lookups in two memory accesses, SODA 2005 Ross – Efficient hash probes on modern processors, Tech. Rep. , IBM 2006 Sundar – A lower bound on the cell probe complexity of the dictionary problem, unpublished manuscript 1993 Zukowski, Héman, Boncz – Architecture conscious hashing, Da. Mo. N 2006 22