Local features detection and description Devi Parikh Disclaimer
- Slides: 55
Local features: detection and description Devi Parikh Disclaimer: Most slides have been borrowed from Kristen Grauman, who may have borrowed some of them from others. Any time a slide did not already have a credit on it, I have credited it to Kristen. So there is a chance some of these credits are inaccurate. Slide credit: Kristen Grauman 1
Announcements • Project proposals – Due on Wednesday • PS 3 out – Due in <3 weeks (October 24 th) Slide credit: Kristen Grauman 2
Topics overview • Intro • Multiple views and motion – Local invariant features • Features & filters – – – Filters Gradients Edges Blobs/regions Local invariant features • Grouping & fitting • Recognition • Video processing Slide credit: Kristen Grauman 3
Last time • Detecting corner-like points in an image Slide credit: Kristen Grauman 4
Today • Local invariant features – Detection of interest points • (Harris corner detection) • Scale invariant blob detection: Lo. G – Description of local patches • SIFT : Histograms of oriented gradients 5 Slide credit: Kristen Grauman
Local features: main components 1) Detection: Identify the interest points 2) Description: Extract vector feature descriptor surrounding each interest point. 3) Matching: Determine correspondence between descriptors in two views Kristen Grauman 6
Properties of the Harris corner detector Rotation invariant? Yes Scale invariant? 7 Slide credit: Kristen Grauman
Properties of the Harris corner detector Rotation invariant? Scale invariant? All points will be classified as edges Slide credit: Kristen Grauman Yes No Corner ! 8
Scale invariant interest points How can we independently select interest points in each image, such that the detections are repeatable across different scales? 9 Slide credit: Kristen Grauman
Automatic scale selection Intuition: • Find scale that gives local maxima of some function f in both position and scale. f Image 1 s 1 region size f Slide credit: Kristen Grauman Image 2 s 2 region size 10
What can be the “signature” function? 11 Slide credit: Kristen Grauman
Recall: Edge detection f Edge Derivative of Gaussian Edge = maximum of derivative 12 Source: S. Seitz
Recall: Edge detection f Edge Second derivative of Gaussian (Laplacian) Edge = zero crossing of second derivative 13 Source: S. Seitz
From edges to blobs • Edge = ripple • Blob = superposition of two ripples maximum Spatial selection: the magnitude of the Laplacian response will achieve a maximum at the center of the blob, provided the scale of the Laplacian is 14 “matched” to the scale of the blob Slide credit: Lana Lazebnik
Blob detection in 2 D Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2 D 15 Slide credit: Kristen Grauman
Blob detection in 2 D: scale selection filter scales Laplacian-of-Gaussian = “blob” detector Bastian Leibe img 1 img 2 16 img 3
Blob detection in 2 D We define the characteristic scale as the scale that produces peak of Laplacian response characteristic scale 17 Slide credit: Lana Lazebnik
Example Original image at ¾ the size Kristen Grauman 18
Original image at ¾ the size 19 Kristen Grauman
20 Kristen Grauman
21 Kristen Grauman
22 Kristen Grauman
23 Kristen Grauman
24 Kristen Grauman
Scale invariant interest points Interest points are local maxima in both position and scale. s 5 s 4 scale s 3 s 2 s 1 Squared filter response maps Slide credit: Kristen Grauman List of (x, y, σ) 25
Scale-space blob detector: Example T. Lindeberg. Feature detection with automatic scale selection. IJCV 1998. Slide source: Kristen Grauman 26
Scale-space blob detector: Example 27 Image credit: Lana Lazebnik
Technical detail We can approximate the Laplacian with a difference of Gaussians; more efficient to implement. (Laplacian) (Difference of Gaussians) 28 Slide credit: Kristen Grauman
Local features: main components 1) Detection: Identify the interest points 2) Description: Extract vector feature descriptor surrounding each interest point. 3) Matching: Determine correspondence between descriptors in two views 29 Slide credit: Kristen Grauman
Geometric transformations e. g. scale, translation, rotation Slide credit: Kristen Grauman 30
Photometric transformations Figure from T. Tuytelaars ECCV 2006 tutorial Slide credit: Kristen Grauman 31
Raw patches as local descriptors The simplest way to describe the neighborhood around an interest point is to write down the list of intensities to form a feature vector. But this is very sensitive to even small shifts, rotations. Slide credit: Kristen Grauman 32
SIFT descriptor [Lowe 2004] • Use histograms to bin pixels within sub-patches according to their orientation. 0 Slide credit: Kristen Grauman 2 p Why subpatches? Why does SIFT have some illumination invariance? 33
Making descriptor rotation invariant CSE 576: Computer Vision • Rotate patch according to its dominant gradient orientation • This puts the patches into a canonical orientation. 34 Slide credit: Kristen Grauman Image from Matthew Brown
SIFT descriptor [Lowe 2004] • Extraordinarily robust matching technique • Can handle changes in viewpoint • Up to about 60 degree out of plane rotation • Can handle significant changes in illumination • Sometimes even day vs. night (below) • • Fast and efficient—can run in real time Lots of code available • http: //people. csail. mit. edu/albert/ladypack/wiki/index. php/Known_implementations_of_SIFT 35 Steve Seitz
Example NASA Mars Rover images 36 Slide credit: Kristen Grauman
Example Slide credit: Kristen Grauman NASA Mars Rover images with SIFT feature matches Figure by Noah Snavely 37
SIFT properties • Invariant to – Scale – Rotation • Partially invariant to – Illumination changes – Camera viewpoint – Occlusion, clutter Slide credit: Kristen Grauman 38
Local features: main components 1) Detection: Identify the interest points 2) Description: Extract vector feature descriptor surrounding each interest point. 3) Matching: Determine correspondence between descriptors in two views 39 Slide credit: Kristen Grauman
Matching local features Kristen Grauman 40
Matching local features ? Image 1 Image 2 To generate candidate matches, find patches that have the most similar appearance (e. g. , lowest SSD) Simplest approach: compare them all, take the closest (or closest k, or within a thresholded distance) Kristen Grauman 41
Ambiguous matches ? ? Image 1 Image 2 At what SSD value do we have a good match? To add robustness to matching, can consider ratio : distance to best match / distance to second best match If low, first match looks good. If high, could be ambiguous match. Kristen Grauman 42
Matching SIFT Descriptors • Nearest neighbor (Euclidean distance) • Threshold ratio of nearest to 2 nd nearest descriptor 43 Slide credit: Kristen Grauman Lowe IJCV 2004
Recap: robust feature-based alignment 44 Source: L. Lazebnik
Recap: robust feature-based alignment • Extract features 45 Source: L. Lazebnik
Recap: robust feature-based alignment • • Extract features Compute putative matches 46 Source: L. Lazebnik
Recap: robust feature-based alignment • • • Extract features Compute putative matches Loop: • Hypothesize transformation T (small group of putative matches that are related by T) 47 Source: L. Lazebnik
Recap: robust feature-based alignment • • • Extract features Compute putative matches Loop: • • Hypothesize transformation T (small group of putative matches that are related by T) Verify transformation (search for other matches consistent with T) 48 Source: L. Lazebnik
Recap: robust feature-based alignment • • • Extract features Compute putative matches Loop: • • Hypothesize transformation T (small group of putative matches that are related by T) Verify transformation (search for other matches consistent with T) 49 Source: L. Lazebnik
Applications of local invariant features • • Wide baseline stereo Motion tracking Panoramas Mobile robot navigation 3 D reconstruction Recognition … 50 Slide credit: Kristen Grauman
Automatic mosaicing http: //www. cs. ubc. ca/~mbrown/autostitch. html 51 Slide credit: Kristen Grauman
Wide baseline stereo [Image from T. Tuytelaars ECCV 2006 tutorial] Slide credit: Kristen Grauman 52
Recognition of specific objects, scenes Schmid and Mohr 1997 Rothganger et al. 2003 Kristen Grauman Sivic and Zisserman, 2003 Lowe 2002 53
Summary • Interest point detection – Harris corner detector – Laplacian of Gaussian, automatic scale selection • Invariant descriptors – Rotation according to dominant gradient direction – Histograms for robustness to small shifts and translations (SIFT descriptor) 54
Questions? 55 Slide credit: Devi Parikh
- Vqa
- Devi parikh
- Devi
- Capitalideasonline
- Mahasweta devi the hunt
- Detection of local attraction
- Io devo tu devi
- Ma leg de tine pamantule
- Chandi prasad bhatt quotes
- Aditi devi bhava
- Devi pratami
- Sadyojatam prapadyami
- Khiloni devi dharamshala pitampura
- Devi personality type
- R latha devi
- Citate din maitreyi
- Istituto comprensivo albaro
- Matematta
- Shinta devi
- Local vertical local horizontal frame
- A think local act local multicountry type of strategy
- A "think local, act local" multidomestic type of strategy
- Two drawbacks of a think local act local
- Object recognition from local scale-invariant features
- The views expressed disclaimer
- Disclaimer the views and opinions expressed
- The views and opinions expressed disclaimer abs-cbn
- Radar acronym
- Checksum in computer networks with example
- Hazard detection and resolution in computer architecture
- Hazard detection and resolution
- Deadlock detection and recovery
- Ids sensors
- Error detection and correction in data link layer
- What is crc
- Collision detection and resolution
- Collide and slide algorithm
- National breast and cervical cancer early detection program
- Starvation deadlock
- National breast and cervical cancer early detection program
- Deadlock prevention avoidance and detection
- Error detection and correction
- Edc error
- Difference between error detection and error correction
- Error correction in computer networks
- Crc example
- Local maximum and minimum vs. absolute maximum and minimum
- State and local taxes and spending
- Disclaimer for sensitive content
- Forward-looking statement disclaimer example
- All opinions expressed disclaimer
- Bowed string instrument
- Forward-looking statement disclaimer example
- Hacking disclaimer
- Disclaimer interview
- Disclaimer trust diagram