Presidents Day Lecture Advanced Nearest Neighbor Search Advanced

President’s Day Lecture: Advanced Nearest Neighbor Search [Advanced Algorithms, Spring’ 17]

Announcements � Evaluation on Course. Works � If you think homework is too easy (or too hard): � mark 2 “appropriateness of workload”

Time-Space Trade-offs (Euclidean) space query Space time Time Comment Reference [Ind’ 01, Pan’ 06] low high [AI’ 06] [IM’ 98, DIIM’ 04] [AI’ 06] medium [MNP’ 06, OWZ’ 11] ω(1) memory lookups [PTW’ 08, PTW’ 10] p oku o l m e 1 m high low ω(1) memory lookups [KOR’ 98, IM’ 98, Pan’ 06] [AIP’ 06]

[Indyk’ 01, Panigrahy’ 06] � Sample a few buckets in the same hash table! 4

Near-linear Space � 5

Beyond LSH Space Time Exponent Hamming space Reference [IM’ 98] [MNP’ 06, OWZ’ 11] LSH [AINR’ 14, AR’ 15] Euclidean space [AI’ 06] [MNP’ 06, OWZ’ 11] [AINR’ 14, AR’ 15] 6 LSH

New approach? Data-dependent hashing �A random hash function, chosen after seeing the given dataset � Efficiently computable 7

Construction of hash function [A. -Indyk-Nguyen-Razenshteyn’ 14, A. -Razenshteyn’ 15] � Two components: � Nice geometric structure � Reduction to such structure 8 has better LSH data-dependent

Nice geometric structure � 9

Alg 1: Hyperplanes � [Charikar’ 02] 10

Alg 2: Voronoi � [A. -Indyk-Nguyen-Razenshteyn’ 14] based on [Karger-Motwani-Sudan’ 94] 11

Hyperplane vs Voronoi � 12

Reduction to nice structure (very HL) � Idea: iteratively decrease the radius of minimum enclosing ball OR make more isotopic � Algorithm: � find Why ok? dense clusters � with smaller radius � large fraction of points � recurse on dense clusters � apply Voronoi. LSH on the rest � recurse on each “cap” � eg, dense clusters might reappear 13 *picture not to scale & dimension

Hash function � Described 14 by a tree (like a hash table) *picture not to scale&dimension

Dense clusters � ?

Tree recap � 16

NNS: conclusion � 1. Via sketches � 2. Locality Sensitive Hashing � Random space partitions � Better space bound � Even � 3. near-linear! Data-dependent hashing even better � Used in practice a lot these days 17