Semantic Robot Vision Challenge Ashutosh Kulkarni Shashank Senapaty

Semantic Robot Vision Challenge Ashutosh Kulkarni Shashank Senapaty Priyanka Singh March 17, 2008

Problem Statement Semantic Robot Vision Challenge (SRVC): Research competition designed to push the state of the art in image understanding and automatic acquisition of knowledge from large unstructured databases of images (such as those generally found on the web). Use web to find image examples of objects Identify these objects in a room environment March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 2

Approach Standard learning appearance models do not work Unreliable training images CD “Hey Eugene” by Pink Martini Solution: Rely on matching local features March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 3

Basic Algorithm Matched SIFT features between training and test images March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 4

Bad (!) training images CD “Hey Eugene” by Pink Martini 2. jpg March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 5

Ranking (Google image search) March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 6

Ranking (within group similarity) March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 7

Problem with similarity based ranking Needed: DVD of Shrek 1 March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 8

Elimination and Sub-classification Weaker notion of a “good” training image – An image which matches with at least one other image from the class. Remove images that do not match with any other image in the class March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 9

New Ranking results Good Images -- Class 1: Good Images --Class 2: March 17, 2008 Eliminated images: Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 10

But everything’s Harry Potter! False Positives: March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 11

RANSAC Throw away outliers Matching coefficient: Number of inliers / Total number of SIFT matches March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 12

Results after RANSAC March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 13

References: SRVC : http: //www. semantic-robot-vision-challenge. org/ Scott Helmer, David Meger, Per-Erik Forss´en, Tristram Southey, Sancho Mc. Cann, Pooyan Fazli, James J. Little, David G. Lowe. The UBC Semantic Robot Vision System. Webpage: http: //www. cs. ubc. ca/~perfo/abstracts/hmfsfll 07. html Scott Helmer, David Meger, Per-Erik Forss´en, Sancho Mc. Cann, Tristram Southey, Matthew Baumann, Kevin Lai, Bruce Dow, James J. Little, David G. Lowe. Curious George: The UBC Semantic Robot Vision System. Webpage: http: //www. cs. ubc. ca/~perfo/abstracts/hmfmsbldll 07. html Li-Jia Li, Gang Wang and Li Fei-Fei. OPTIMOL: automatic Object Picture collec. Tion via Incremental Model Learning. CVPR 2007. [PDF] SIFT implementation: http: //web. engr. oregonstate. edu/~hess/index. html Web crawling resources: http: //search. cpan. org/%7 Egrousse/WWW-Google. Images-0. 6. 4/lib/WWW/Google/Images. pm March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 14

Questions? March 17, 2008 Ashutosh Kulkarni • Shashank Senapaty • Priyanka Singh 15