Search as You Type From Research to Commercialization

Search as You Type: From Research to Commercialization Chen Li UC Irvine DBRank Workshop, August 27, 2012, Istanbul, Turkey 1

Q: Why is DBRank in Istanbul? Topkapı Palace Top-k api 2

Personal Journey: 2001 … 3

Data Integration Problems? Talking to medical doctors… 4

Example Table R Name SSN Table S Addr Name SSN Addr Jack Lemmon 430 -871 -8294 Maple St Ton Hanks 234 -162 -1234 Main Street Harrison Ford 292 -918 -2913 Culver Blvd Kevin Spacey 928 -184 -2813 Frost Blvd Tom Hanks 234 -762 -1234 Main St Jack Lemon 430 -817 -8294 Maple Street … … … Q: Find records from different datasets that could be the same entity Chen Li 5

Challenges n How to define good similarity functions? n How to do matching efficiently? Chen Li 6

The Flamingo Package http: //flamingo. ics. uci. edu/

Make an impact? 8

UCI People Search Chen Li 9

Psearch (2008) : 2 stories Chen Li 10

Fuzzy search 11

Location-based search www. omniplaces. com 12

Bimaple Incorporated (2008) Chen Li 13

Query suggestions Chen Li 14

Instant Search Chen Li 15

Different from Google Instant

Advantages of instant fuzzy search Correct errors Save time Mobile friendly Fat fingers! Chen Li 17

Comparing Two Search Systems UCI Directory Search � Traditional Keyword Search on UCI People Directory � Frequently used by UCI community 18

User Behavior Analysis Log Structure � Log Record, Click Record, Query, Peak Query Log Record Peak Query Click IP Timestamp Chen Li Queries 19

User Behavior Analysis 61% • • • 19. 4% 12. 2% 7. 2% 96% queries, 4% clicks 35% of sessions have click records Median of session durations is 3 sec Average peak query length is 8. 89 chars Average number of queries in a session is 9. 16 Chen Li 20

Benefits of Instant Search Median of Session Durations PSearch UCI Directory Search 3 s 5 s Chen Li 21

Challenges Speed � “ 100 ms rule” � Prefix matching � Fuzzy matching Quality Chen Li 22

Database built-in full-text search Example (Oracle) SELECT SCORE(1) score, id, name FROM Products WHERE CONTAINS(doc, ‘iphone', 1) > 0 ORDER BY SCORE(1) DESC; Limitations � Speed � Ranking

Open-source packages Limitations � Weak support of fuzzy search � Poor ranking in instant search

Results Efficient indexing and searching (WWW 09, SIGMOD 09, Bioinformatics 09, ICDE 10, VLDBJ 11) Ranking (SIGIR 12) Log Analysis (Web. DB 12) Map search (SSDBM 11) Chen Li 25

Index structures Trie ID Record r 0 graph icdm g i r 1 graph group lui r c r 2 gray icde liu r 3 graph icde lin lui r 4 graph group icdm lin liu r 5 graph gray gross icdm lin liu r 6 gray group icdm lin liu r 7 gray gross group icde lin r 8 gross icde liu r 9 icdm liu a p o y h l i d s u s p e m u n u i Inverted Index Chen Li 26

Fuzzy-search Algorithm: Overview

Multi-Prefix Intersection Q = vldb li ID Record 1 Li data… 2 data… 3 data Lin… 4 Lu Lin Luis… 5 Liu… 6 VLDB Lin data… t $ n u $ i d 7 8 VLDB… Li VLDB… a 1 8 $ $ 4 s b 3 4 6 5 $ $ 4 6 7 8 d l a $ 1 2 3 6 v i u l

Multi-Prefix Intersection: Method 1 ID Record 1 Li data… 2 data… 3 data Lin… a 4 Lu Lin Luis… t $ n u $ i d 5 Liu… 6 VLDB Lin data… a $ $ 4 s b 7 VLDB… 1 8 5 $ $ 8 Li VLDB… 3 4 6 7 8 Space cost Time cost d $ 1 2 Inverted index 3 Union + intersection 6 Q = vldb li l v i u l li 1 3 4 5 6 8 vldb 6 7 8 More efficient intersection approaches… 6 8

Multi-Prefix Intersection: Method 2 ID Record Forward List 1 Li data… 1 2 2 data… 1 3 data Lin… 1 3 4 Lu Lin Luis… 3 5 6 Liu… 4 VLDB Lin data… 1 3 7 7 VLDB… 7 8 Li VLDB… 2 7 6 Read each 7 8 [2, 4] Q = vldb li [1, 7] d [1, 1] l a [1, 1] t [1, 1] i $2 a [1, 1] 1 8 $1 1 2 3 Verify/Probe 6 [2, 6] [2, 4] u [5, 6] v [7, 7] l [7, 7] n [3, 3]u [4, 4] $ 5 i [6, 6] d [7, 7] $3 $4 s [6, 6] b [7, 7] 3 4 6 5 4 $6 $7 4 6 7 8 Space cost Inverted + forward index Time cost Probing forward lists

Ranking Aggregate Query icd e grap h Score(graph) Score(icde) li Max Score(lin) Score(liu) Record graph, gray, gross, icde, lin, liu Chen Li 31

Multi-Prefix Intersection {graph, icde, li} k=1 g i r c a p o y h graph l i d s u s p e m icde Too many candidates u n u i lin Chen Li liu 32

Optimization 1: Heap-based Method Aggregate Get. Max() icde graph Max Heap lin liu Chen Li 33

Optimization 2: Top-k List-Merging Algorithm Example: Threshold algorithm Sorted Access T = 15 Random Access Early termination Chen Li 34

Random Access Using Forward Index ID “graph” “icde” “li” 7 0 7? ID Forward list r 0 <1, 2> <5, 3> r 1 <1, 3> <1, 9> <9, 6> r 2 <2, 9> <5, 2> <8, 3> r 3 <1, 4> <5, 2> <7, 9> <9, 4> p y s u e m r 4 <1, 7> <4, 3> <6, 9> <7, 2> <8, 7> h 2 s p 5 6 … 1 3 4 … [7, 9] [1, 4] i [5, 6] g [1, 4] r [1, 2] a Chen Li c [5, 6] [3, 4] [1, 1] [2, 2] [3, 3] [7, 8] o [4, 4] i l [9, 9] u d [5, 6] n u i 7 8 9 35

Ranking Framework (Fuzzy matching) Aggregate Query grap h Score(graph) icd e li Max Sim(li, i) Sim(icde, icdm) *Score(lin) Score(liu) *Score(icdm) Record graph, gray, icdm, gross, lin, liu Chen Li 36

Computing Similar Prefixes {graph, icde, li}, similarity threshold τ=0. 45 g i r c a p o y h l i d s u s p e u n u i m Chen Li 37

Top-k Algorithm sum Get. Max() graph Get. Max() li icde Max Heap × 1 × 0. 5 Max Heap × 0. 5 × 1 × 0. 5 similarity lui graph icde icdm Chen Li lin liu 38

Other interesting problems www. omniplaces. com 39

Current status of Bimaple n n n Mainly focusing on product development (C++) With angel investment Has customers Chen Li 40

Chen Li 41

1. Different challenges Technology Products Chen Li Business 42

2: Hands-on experiences Chen Li 43

3: More systems challenges E. g. : Updates; Multi-threading … Chen Li 44

4: Balancing Chen Li 45

5: Getting support Chen Li 46

6: Make a difference Chen Li 47

Thank you Supported by NSF (US), NIH (US), and NSF (China). Chen Li 48