Computer Vision Fitting Marc Pollefeys COMP 256 Some

  • Slides: 59
Download presentation
Computer Vision Fitting Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth,

Computer Vision Fitting Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, T. Darrel, A. Zisserman, . . .

Tentative class schedule Computer Vision Jan 16/18 - Introduction Jan 23/25 Cameras Radiometry Sources

Tentative class schedule Computer Vision Jan 16/18 - Introduction Jan 23/25 Cameras Radiometry Sources & Shadows Color Feb 6/8 Linear filters & edges Texture Feb 13/15 Multi-View Geometry Stereo Feb 20/22 Optical flow Project proposals Affine Sf. M Projective Sf. M Camera Calibration Segmentation Mar 13/15 Springbreak Mar 20/22 Fitting Prob. Segmentation Mar 27/29 Silhouettes and Photoconsistency Linear tracking Apr 3/5 Project Update Non-linear Tracking Apr 10/12 Object Recognition Apr 17/19 Range data Final project Jan 30/Feb 1 Feb 27/Mar 1 Mar 6/8 2 Apr 24/26

Computer Vision Last time: Segmentation • Group tokens into clusters that fit together –

Computer Vision Last time: Segmentation • Group tokens into clusters that fit together – foreground-background – cluster on intensity, color, texture, location, … • K-means • graph-based 3

Computer Vision Fitting • Choose a parametric object/some objects to represent a set of

Computer Vision Fitting • Choose a parametric object/some objects to represent a set of tokens • Most interesting case is when criterion is not local – can’t tell whether a set of points lies on a line by looking only at each point and the next. 4 • Three main questions: – what object represents this set of tokens best? – which of several objects gets which token? – how many objects are there? (you could read line for object here, or circle, or ellipse or. . . )

Computer Vision Hough transform : straight lines implementation : 5 1. the parameter space

Computer Vision Hough transform : straight lines implementation : 5 1. the parameter space is discretised 2. a counter is incremented at each cell where the lines pass 3. peaks are detected

Computer Vision Hough transform : straight lines problem : unbounded parameter domain vertical lines

Computer Vision Hough transform : straight lines problem : unbounded parameter domain vertical lines require infinite a alternative representation: Each point will add a cosine function in the ( , ) parameter space 6

Computer Vision tokens votes 7

Computer Vision tokens votes 7

Computer Vision Hough transform : straight lines Square : 8 Circle :

Computer Vision Hough transform : straight lines Square : 8 Circle :

Computer Vision 9 Hough transform : straight lines

Computer Vision 9 Hough transform : straight lines

Computer Vision Mechanics of the Hough transform • Construct an array representing , d

Computer Vision Mechanics of the Hough transform • Construct an array representing , d • For each point, render the curve ( , d) into this array, adding one at each cell • Difficulties – how big should the cells be? (too big, and we cannot distinguish between quite different lines; too small, and noise causes lines to be missed) 10 • How many lines? – count the peaks in the Hough array • Who belongs to which line? – tag the votes • Hardly ever satisfactory in practice, because problems with noise and cell size defeat it

Computer Vision tokens 11 votes

Computer Vision tokens 11 votes

Computer Vision 12

Computer Vision 12

Computer Vision 13

Computer Vision 13

Computer Vision 14

Computer Vision 14

Computer Vision 15 Cascaded hough transform Tuytelaars and Van Gool ICCV‘ 98

Computer Vision 15 Cascaded hough transform Tuytelaars and Van Gool ICCV‘ 98

Computer Vision standard least-squares Line fitting can be max. likelihood - but choice of

Computer Vision standard least-squares Line fitting can be max. likelihood - but choice of model is important total least-squares 16

Computer Vision Who came from which line? • Assume we know how many lines

Computer Vision Who came from which line? • Assume we know how many lines there are - but which lines are they? – easy, if we know who came from which line • Three strategies – Incremental line fitting – K-means – Probabilistic (later!) 17

Computer Vision 18

Computer Vision 18

Computer Vision 19

Computer Vision 19

Computer Vision 20

Computer Vision 20

Computer Vision 21

Computer Vision 21

Computer Vision 22

Computer Vision 22

Computer Vision 23

Computer Vision 23

Computer Vision 24

Computer Vision 24

Computer Vision 25

Computer Vision 25

Computer Vision 26

Computer Vision 26

Computer Vision 27

Computer Vision 27

Computer Vision 28

Computer Vision 28

Computer Vision 29

Computer Vision 29

Computer Vision 30

Computer Vision 30

Computer Vision 31

Computer Vision 31

Computer Vision Robustness • As we have seen, squared error can be a source

Computer Vision Robustness • As we have seen, squared error can be a source of bias in the presence of noise points – One fix is EM - we’ll do this shortly – Another is an M-estimator • Square nearby, threshold far away – A third is RANSAC • Search for good points 32

Computer Vision 33

Computer Vision 33

Computer Vision 34

Computer Vision 34

Computer Vision 35

Computer Vision 35

Computer Vision 36

Computer Vision 36

Computer Vision M-estimators • Generally, minimize where 37 is the residual

Computer Vision M-estimators • Generally, minimize where 37 is the residual

Computer Vision 38

Computer Vision 38

Computer Vision 39

Computer Vision 39

Computer Vision 40

Computer Vision 40

Computer Vision 41 Too small

Computer Vision 41 Too small

Computer Vision 42 Too large

Computer Vision 42 Too large

Computer Vision 43

Computer Vision 43

Computer Vision RANSAC • Choose a small • Issues subset uniformly at – How

Computer Vision RANSAC • Choose a small • Issues subset uniformly at – How many times? random • Often enough that we are likely to have a • Fit to that good line • Anything that is close – How big a subset? to result is signal; all • Smallest possible others are noise – What does close mean? • Refit • Depends on the problem • Do this many times – What is a good line? and choose the best • One where the number of nearby points is so big it is unlikely to be all outliers 44

Computer Vision 45

Computer Vision 45

Computer Vision Distance threshold Choose t so probability for inlier is α (e. g.

Computer Vision Distance threshold Choose t so probability for inlier is α (e. g. 0. 95) • Often empirically • Zero-mean Gaussian noise σ then follows distribution with m=codimension of model (dimension+codimension=dimension space) 46 Codimension Model t 1 line, F 3. 84σ2 2 H, P 5. 99σ2 3 T 7. 81σ2 2

Computer Vision How many samples? Choose N so that, with probability p, at least

Computer Vision How many samples? Choose N so that, with probability p, at least one random sample is free from outliers. e. g. p=0. 99 proportion of outliers e 47 s 2 3 4 5 6 7 8 5% 2 3 3 4 4 4 5 10% 3 4 5 6 7 8 9 20% 5 7 9 12 16 20 26 25% 6 9 13 17 24 33 44 30% 7 11 17 26 37 54 78 40% 50% 11 17 19 35 34 72 57 146 97 293 163 588 272 1177

Computer Vision Acceptable consensus set? • Typically, terminate when inlier ratio reaches expected ratio

Computer Vision Acceptable consensus set? • Typically, terminate when inlier ratio reaches expected ratio of inliers 48

Computer Vision Adaptively determining the number of samples e is often unknown a priori,

Computer Vision Adaptively determining the number of samples e is often unknown a priori, so pick worst case, e. g. 50%, and adapt if more inliers are found, e. g. 80% would yield e=0. 2 – N=∞, sample_count =0 – While N >sample_count repeat • • Choose a sample and count the number of inliers Set e=1 -(number of inliers)/(total number of points) Recompute N from e Increment the sample_count by 1 – Terminate 49

Computer Vision RANSAC for Fundamental Matrix Step 1. Extract features Step 2. Compute a

Computer Vision RANSAC for Fundamental Matrix Step 1. Extract features Step 2. Compute a set of potential matches Step 3. do Step 3. 1 select minimal sample (i. e. 7 matches) Step 3. 2 compute solution(s) for F Step 3. 3 determine inliers (verify hypothesis) (generate hypothesis) until (#inliers, #samples)<95% Step 4. Compute F based on all inliers Step 5. Look for additional matches Step 6. Refine F based on all correct matches 50 #inliers 90% 80% 70% 60% 50% #samples 5 13 35 106 382

Computer Vision Randomized RANSAC for Fundamental Matrix Step 1. Extract features Step 2. Compute

Computer Vision Randomized RANSAC for Fundamental Matrix Step 1. Extract features Step 2. Compute a set of potential matches Step 3. do Step 3. 1 select minimal sample (i. e. 7 matches) (generate Step 3. 2 compute solution(s) for F hypothesis) Step 3. 3 Randomize verification 3. 3. 1 verify if inlier (verify hypothesis) while hypothesis is still promising while (#inliers, #samples)<95% Step 4. Compute F based on all inliers Step 5. Look for additional matches Step 6. Refine F based on all correct matches 51

Computer Vision Example: robust computation from H&Z Interest points (500/image) #in 1 -e adapt.

Computer Vision Example: robust computation from H&Z Interest points (500/image) #in 1 -e adapt. N 6 2% 20 M 10 3% 2. 5 M 44 16% 6, 922 58 21% 2, 291 73 26% 911 151 56% 43 (640 x 480) Putative correspondences (268) (Best match, SSD<20, ± 320) Outliers (117) (t=1. 25 pixel; 43 iterations) Inliers (151) Final inliers (262) 52 (2 MLE-inlier cycles; d =0. 23→d =0. 19; Iter. Lev-Mar=10)

Computer Vision More on robust estimation • LMed. S, an alternative to RANSAC (minimize

Computer Vision More on robust estimation • LMed. S, an alternative to RANSAC (minimize Median residual in stead of maximizing inlier count) • Enhancements to RANSAC – Randomized RANSAC – Sample ‘good’ matches more frequently –… • RANSAC is also somewhat robust to bugs, sometimes it just takes a bit longer… 53

Computer Vision RANSAC for quasi-degenerate data • Often data can be almost degenerate e.

Computer Vision RANSAC for quasi-degenerate data • Often data can be almost degenerate e. g. 337 matches on plane, 11 off plane planar points only provide 6 in stead of 8 linearly independent equations for F • RANSAC gets confused by quasi-degenerate data Probability of valid non-degenerate sample 54 Probability of success for RANSAC (aiming for 99%)

Computer Vision 55 RANSAC for quasi-degenerate data (Frahm and Pollefeys, CVPR 06) (QDEGSAC) QDEGSAC

Computer Vision 55 RANSAC for quasi-degenerate data (Frahm and Pollefeys, CVPR 06) (QDEGSAC) QDEGSAC estimates robust rank of datamatrix (i. e. large % of rows approx. fit in a lower dimensional subspace)

Computer Vision Experiments for computing F matrix Comparison to Chum et al. (CVPR’ 05)

Computer Vision Experiments for computing F matrix Comparison to Chum et al. (CVPR’ 05) Data courtesy of Chum et al. 56 QDEGSAC (and DEGENSAC) are successful at dealing with quasi-degenerate data

Computer Vision Experiments for computing P matrix Similar results for H, Q, … QDEGSAC

Computer Vision Experiments for computing P matrix Similar results for H, Q, … QDEGSAC is general and can deal with other robust estimation problems QDEGSAC does not require to know which degeneracies can occur 57

Computer Vision Fitting curves other than lines • In principle, an easy generalisation –

Computer Vision Fitting curves other than lines • In principle, an easy generalisation – The probability of obtaining a point, given a curve, is given by a negative exponential of distance squared 58 • In practice, rather hard – It is generally difficult to compute the distance between a point and a curve

Computer Vision Next class: Segmentation and Fitting using Probabilistic Methods Missing data: EM algorithm

Computer Vision Next class: Segmentation and Fitting using Probabilistic Methods Missing data: EM algorithm Model selection 59 Reading: Chapter 16