Gustavo Carneiro Set of Matching Problems Wide baseline

Gustavo Carneiro

Set of Matching Problems Wide baseline Matching Visual Object Recognition Visual Class Recognition

Set of Matching Problems 1 - Design a feature space that facilitates certain matching problems SIFT [Lowe, ICCV 09] Shape Context [Belongie et al. PAMI 02] HOG [Dalal & Triggs, CVPR 05]

Set of Matching Problems 2 - Given a matching problem, and a set of feature spaces, combine them in order to minimize probability of error (mismatches) [Varma & Ray, CVPR 07] SIFT Target Matching Problem Shape Context HOG

Set of Matching Problems 3 - Given a matching problem, find the feature space and respective parameters θ that minimizes probability of error (mismatches) [Hua et al. ICCV 07] Target Matching Problem Feature Transform 21(θ*) Transform (θ) Transform 12(θ*) (θ)

Set of Matching Problems 4 - Given future unknown matching problems, find the feature space that minimizes probability of error (mismatches) Target Matching Feature Transform Problem 1 Transform Matching Problem 1 Feature Transform Matching Problem 2 Target Matching Problem 2 Feature Transform Matching Problem 5 Matching Problem 3 Feature Transform Matching Problem 4

The Universal Feature Transform • Solve random and simple matching problems • The more matching problems solved, the easier it will be to solve new problems • Restriction: problems should be in similar feature ranges and similar class statistics

(Linear) Distance Metric Learning [Chopra et al. CVPR 05, Goldberger et al. NIPS 04, Weinberger & Saul JMLR 09] • Image patches: • Linear transform: • Distance in T space:

(Non-Linear) Distance Metric Learning [Sugiyama JMLR 07] • Rewrite S(b) and S(w): • By taking the following transformation: • Generalized Eigenvalue Problem: Dot product replaced by non-linear kernel function • Feature Transform

Linear vs Non-linear DML R A INE L NON LINE AR Points from the same class collapse and are far from each other Points not belonging to any class collapse at the origin

Intuition • Train several feature transforms – Random matching problems • Aggregate distances [Breiman 01]: – • Threshold-based classifier –

Aggregated distances Intuition ROC Unkown target problem Small dist. Large dist. T Random training problem 1

Aggregated distances Intuition ROC Unkown target problem Small dist. Largedist. T Random training problem 2

Toy Example • Combing 100 feature spaces. . . NLMSL trained UFT Original Error decreases with number of feature spaces No matter the error for each space

Experiments • Dataset of for training [Winder & Brown, CVPR 07]: – – – Backprojecting 3 D points to 2 D images from scene reconstructions Variations in scene location, brightness and partial occlusion Similar pre-processing of [Winder & Brown, CVPR 07] Train: all patch classes from Trevi & Yosemite dataset Test: 50 K matching and 50 K non-matching pairs from Notre Dame dataset

Experiments • Using cross validation – 50 training classes for training each feature space – 50 training feature spaces UFT (2. 28%) SIFT (6. 3%) @95% TP Error decreases with number of feature spaces No matter the error in each space

Experiments • Matching database [Mikolajczyk & Schmid, PAMI’ 05]

Conclusion • Competitive performance • Simple ensemble classifier (can be efficiently implemented) • Adapt to new classification problems (no re-training)

Linear vs Non-linear DML 10 runs, 100 points per class Classifier: threshold matching AR E LIN NON LINE AR Non-linear: low bias, high variance Linear: High bias, low variance

Combining Feature Spaces • Breiman’s idea about ensemble classifiers [Breiman 01]: – combine low-bias, high-variance (unstable) classifiers to produce low-bias, low-variance classifiers. • Distance