What is a Good Nearest Neighbors Algorithm for

What is a Good Nearest Neighbors Algorithm for Finding Similar Patches in Images? Neeraj Kumar*, Li Zhang†, Shree K. Nayar* *Columbia University, †University of Wisconsin-Madison http: //www. cs. columbia. edu/CAVE/projects/nnsearch / Motivation Query Patch Results Nearest Neighbor Approaches All methods organize points into a tree structure, with the only difference being the function used to split a set of points into different child nodes (shown below). Performance Evaluation Construction and Search Performance • Cons. cost is number of distance function evaluations • Search speed is improvement over brute-force search kd-Tree Brute-Force Search: 2125 ms Brute-Force Search: 1375 ms vp-Tree Search: 1. 25 ms vp-Tree Search: 3. 43 ms Brute-force search for all patches would take >250 hrs/image! Fast search would greatly speed-up many vision algorithms: • Object Recognition – e. g. , using SIFT [Lowe 2003] • Image Denoising using non-local means [Buades et al. 2005] • Shape Matching using self-similarity [Schechtman & Irani 2007] • Texture Synthesis using image quilting [ Efros & Freeman 2002] The Nearest Neighbors Problem Given a set of points P and query point q, the NN problem is: ε-NN Search: Find set of points PC within distance ε: k-NN Search: Find set of k points PC closest to q: Example results for query point q: dim = split L_pts R_pts argmaxdim[var(ptsdim)] = median(ptsdim) = [ptsdim < split] = [ptsdim > split] PCA Tree axis = pca(pts). eigvec[0] curpts = project(pts, axis) split = median(curpts) L_pts = [curpts < split] R_pts = [curpts > split] Search Speed vs. Distance/Patch Size Exp one Ball Tree pt 1, pt 2 = choose. Ref. Pts(pts) d 1 = d(pts, pt 1) d 2 = d(pts, pt 2) L_pts = [d 1 < d 2] R_pts = [d 2 < d 1] ntia l dr opo ff in e am ce n s an ti hi dist s w l” int ixe po er-p r we e p Fe rag ve “a spe ed Exponential dro poff in spee d Closest k points increasingly distant Vantage Point (vp) Tree pt = choose. Ref. Pt(pts) split = median(d(pts, pt)) L_pts = [d(pts, pt) < split] R_pts = [d(pts, pt) > split] Search Speed vs. Input Set Size Random Images Video Frames es Implementation Tricks ε-NN: p 4, p 6, p 8 2 -NN: p 6, p 8 We use various optimizations on all trees: 4 -NN: p 6 , p 8, p 4, p 3 • Compute Lp norms using lookup tables ila No similarity within images h sim ithin w rity fram Hig • Pre-calculate distances within each leaf node Related Work • Surveys: [Chavez et al. 2001, Shakhnarovich et al. 2006] • Approximate NN: [Arya et al. 1998, Indyk and Motwani 1998] • Nearest Neighbors using L ∞: [Nene and Nayar 1997] • Performance Comparisons: [Mikolajczyk and Matas 2007] • Use priority queues for k-NN searches Evaluation Dataset Conclusions Method kd-Tree PCA Tree Ball Tree k-Means vp-Tree Cons. Perf. Excellent Poor Fair Poor Excellent ε-NN Speed Poor Fair Excellent Good Excellent k-NN Speed Poor Fair Excellent Good Excellent