Image Stitching Panorama Image Stitching Image Stitching Algorithm

Image Stitching

Panorama Image Stitching + +. . . + =

Image Stitching Algorithm Overview 1. Detect feature 2. Match feature 3. Estimate homography using RANSAC 4. Project each image onto the same surface 5. Blend and Crop

Detect Feature (SIFT) 1. Scale Space & DOG Space For each octave of scale space, the initial image is repeatedly convolved with Gaussians to produce the set of scale space images shown on the left. Adjacent Gaussian images are subtracted to produce the difference-of-Gaussian images on the right. After each octave, the Gaussian image is down-sampled by a factor of 2, and the process repeated

Detect Feature (SIFT) 2. Extrema Detection Maxima and minima of the difference-of. Gaussian images are detected by comparing a pixel (marked with X) to its 26 neighbors in 3 × 3 regions at the current and adjacent scales (marked with circles).

Detect Feature (SIFT) 3. Keypoint Then use parabolic interpolation to look for the accurate (x, y, σ) location of the extrema. Reject the points with low contrast (by thresholding pixel value in DOG image) or on the edge (by thresholding principal curvature), to get more distinctive features.

Detect Feature (SIFT) 4. Descriptor Representation For each keypoint detected by the previous procedure, the orientations of its neighbor points will be collected and used to build an orientation histogram, weighted by the magnitude of their gradients, together with a Gaussian kernel centered at the keypoint. Each histogram uses 8 different bins ranging from 0 to 360 degrees. This figure shows a 2 × 2 descriptor array computed from an 8 × 8 set of samples, whereas the experiments in this paper use 4 × 4 descriptors computed from a 16 × 16 sample array.

Match Feature Euclidean distance of descriptor is the distance measure for feature matching between two images. A match is considered not convincing and therefore rejected, if the distances from a point to its closest neighbor and second-closest neighbor are similar.

Estimate homography using RANSAC

Estimate homography using RANSAC Scenario: We’ve got way more matched points than needed to fit the parameters, but we’re not sure which are correct A data set with many outliers for which a line has to be fitted. Fitted line with RANSAC; outliers have no influence on the result.

Estimate homography using RANSAC Algorithm Repeat N times ◦ Randomly select some samples ◦ Fit the model with random samples ◦ See how many other points agree Best estimate is one with most agreement

Estimate homography using RANSAC Repeat N times ◦ ◦ ◦ Choose 4 random potential matches Compute H using normalized DLT Project points from x to x’ for each potentially matching pair Count points with projected distance < t Match validation ◦ The overlapping area within two images shouldn't differ too much ◦ Within the region, a reasonably large portion of matches should be inliers.

Project each image onto the same surface Planar

Project each image onto the same surface Cylindrical http: //graphics. stanford. edu/courses/cs 178 -12/applets/projection. html

Blend = + 1 0

Blend Affect of window size 1 left 1 right 0 0

Blend Affect of window size 1 1 0 0

Blend Good window size: Smooth but not ghosted

Crop Problem : Drift A problem lies in the pairwise alignment process. As you align more and more pairs, the solution may drift so that it is no longer globally consistent.

Crop copy of first image

Reference [1] http: //ppwwyyxx. com/2016/How-to-Write-a-Panorama-Stitcher/#ref [2] Distinctive Image Features From Scale-invariant Keypoints, IJCV 04 [3] https: //courses. engr. illinois. edu/cs 498 dwh/fa 2010/lectures/Lecture%2017%20%20 Photo%20 Stitching. pdf