CSci 6971 Image Registration Lecture 5 FeatureBase Registration

CSci 6971: Image Registration Lecture 5: Feature-Base Registration March 8, 2005 Prof. Charlene Tsai Image Registration Lecture 5

Overview § What is feature-based (point-based) registration? § Feature points § The correspondence problem § Solving for the transformation estimate § Putting it all together: ICP § Discussion and conclusion Image Registration Lecture 5 2

What is Feature-Based Registration? § Images are described as discrete sets of point locations associated with a geometric measurement § Locations may have additional properties such as intensities and orientations § Registration problem involves two parts: § Finding correspondences between features § Estimating the transformation parameters based on these correspondences Image Registration Lecture 5 3

Feature Examples: Range Data § Range image points: § (x, y, z) values § Triangulated mesh § Surface normals are sometimes computed § Notice: § Some information (locations) is determined directly by the sensor (“raw data”) § Some information is inferred from the data Image Registration Lecture 5 4

Feature Examples: Vascular Landmarks § Branching points pulmonary images: § Lung vessels § Airway branches § Retinal image branches and cross-over points § Typically augmented (at least) with orientations of vessels meeting to form landmarks Image Registration Lecture 5 5

Points Along Centers of Vessels and Airways § Airways and vessels modeled as tubular structures § Sample points spaced along center of tubes § Note that the entire tube is rarely used as a unit § Augmented descriptions: § Orientation § Radius Image Registration Lecture 5 6

“Interest” Points § Locations of strong intensity variation in all directions § Augmented with summary descriptions (moments) of surrounding intensity structures § Recent work in making these invariant to viewpoint and illumination. § We’ll discuss interest points during Lectures 15 and 16 Brown and Lowe, Int. Conf. On Computer Vision, 2003 Image Registration Lecture 5 7

Feature Points: Discussion § Many different possible features § Problem is reliably extracting features in all images § This is why more sophisticated features are not used § Feature extraction methods do not use all intensity values § Use of features dominates range-image registration techniques where “features” are provided by the sensor Image Registration Lecture 5 8

Preamble to Feature-Based Registration: Notation § Set of moving image features § Set of fixed image features § Each feature must include a point location in the coordinate system of its image. It may include more § Set of correspondences Image Registration Lecture 5 9

Mathematical Formulation § Error objective function depends on unknown transformation parameters and unknown feature correspondences § Each may depend on the other! § Transformation may include mapping of more than just locations § Distance function, D, could be as simple as the Euclidean distance between location vectors. § We are using the forward transformation model. Image Registration Lecture 5 10

Correspondence Problem § Determine correspondences before estimating transformation parameters § Based on rich description of features § Error prone § Determine correspondences at the same time as estimation of parameters § “Chicken-and-egg” problem § For the next few minutes we will assume a set of correspondences is given and proceed to the estimation of parameters § Then we will return to the correspondence problem Image Registration Lecture 5 11

Example: Estimating Parameters § 2 d point locations: § Similarity transformation: § Euclidean distance: Image Registration Lecture 5 12

Putting This Together Image Registration Lecture 5 13

What Do We Have? § Least-squares objective function § Quadratic function of each parameter § We can § Take the derivative with respect to each parameter § Set the resulting gradient to 0 (vector) § Solve for the parameters through matrix inversion § We’ll do this in two forms: component and matrix/vector Image Registration Lecture 5 14

Component Derivative (a) Image Registration Lecture 5 15

Component Derivative (b) At this point, we’ve dropped the leading factor of 2. It will be eliminated when this is set to 0. Image Registration Lecture 5 16

Component Derivatives tx and ty Image Registration Lecture 5 17

Gathering § Setting each of these equal to 0 we obtain a set of 4 linear equations in 4 unknowns. Gathering into a matrix we have: Image Registration Lecture 5 18

Solving § This is a simple equation of the form § Provided the 4 x 4 matrix X is full-rank (evaluate SVD) we easily solve as Image Registration Lecture 5 19

Matrix Version § We can do this in a less painful way by rewriting the following intermediate expression in terms of vectors and matrices: Image Registration Lecture 5 20

Matrix Version (continued) § This becomes § Manipulating: Image Registration Lecture 5 21

Matrix Version (continued) § Taking the derivative of this w. r. t. the transformation parameters (we didn’t cover vector derivatives, but this is fairly straightforward): § Setting this equal to 0 and solving yields: Image Registration Lecture 5 22

Comparing the Two Versions § Final equations are identical (if you expand the symbols) § Matrix version is easier (once you have practice) and less error prone § Sometimes efficiency requires handcalculation and coding of individual terms Image Registration Lecture 5 23

Resetting the Stage § What we have done: § Features § Error function of transformation parameters and correspondences § Least-squares estimate of transformation parameters for fixed set of correspondences § Next: § ICP: joint estimation of correspondences and parameters Image Registration Lecture 5 24

Iterative Closest Points (ICP) Algorithm § Given an initial transformation estimate 0 § t=0 § Iterate until convergence: § Establish correspondences: § For fixed transformation parameter estimate, t, apply the transformation to each moving image feature and find the closest fixed image feature § Estimate the new transformation parameters, § For the resulting correspondences, estimate t+1 ICP algorithm was developed almost simultaneous by at least 5 research groups in the early 1990’s. Image Registration Lecture 5 25

Finding Correspondences § Map feature into coordinate system of If § Find closest point Image Registration Lecture 5 26

Finding Correspondences (continued) § Enforce unique correspondences § Avoid trivial minima of objective function due to having no correspondences § Spatial data structures needed to make search for correspondences efficient § K-d trees § Digital distance maps § More during lectures 11 -15… Image Registration Lecture 5 27

Initialization and Convergence § Initial estimate of transformation is again crucial because this is a minimization technique § Determining correspondences and estimating the transformation parameters are two separate processes § With Euclidean distance metrics you can show they are working toward the same minimum § In general this is not true § Convergence in practice is sometimes problematic and the correspondences oscillate between points. Image Registration Lecture 5 28

2 d Retinal Example § White = vessel centerline points from one image § Black = vessel centerline points from second image § Yellow line segments drawn between corresponding points § Because of the complexity of the structure, initialization must be fairly accurate Image Registration Lecture 5 29

Comparison Intensity-Based Feature-Based § For a given transformation estimate, we can only find a new, better estimate, not the best estimate, based on the gradient step. § We then need to update the constraints and reestimate § For given set of correspondences, we can directly (leastsquares) estimate the best transformation § BUT, the transformation depends on the correspondences, so we generally need to reestablish the correspondences. Image Registration Lecture 5 30

Summary § § Feature-based registration Feature types and properties Correspondences Least-squares estimate of parameters based on correspondences § ICP § Comparison Image Registration Lecture 5 31

Homework 3 § Ready online. Image Registration Lecture 5 32

Looking Ahead to Lecture 6 § Introduction to ITK and the ITK registration framework. Image Registration Lecture 5 33