CSE 554 Lecture 10 Extrinsic Deformations Fall 2013

CSE 554 Lecture 10: Extrinsic Deformations Fall 2013 CSE 554 Extrinsic Deformations Slide 1

Review Source • Non-rigid deformation – Intrinsic methods: deforming the boundary points – An optimization problem Target Before • Minimize shape distortion • Maximize fit – Example: Laplacian-based deformation CSE 554 After Extrinsic Deformations Slide 2

Extrinsic Deformation • Computing deformation of each point in the plane or volume – Not just points on the boundary curve or surface Credits: Adams and Nistri, BMC Evolutionary Biology (2010) CSE 554 Extrinsic Deformations Slide 3

Extrinsic Deformation • Applications – Registering contents between images and volumes – Interactive spatial deformation CSE 554 Extrinsic Deformations Slide 4

Techniques • Thin-plate spline deformation • Free form deformation • Cage-based deformation CSE 554 Extrinsic Deformations Slide 5

Thin-Plate Spline • Given corresponding source and target points • Computes a spatial deformation function for every point in the 2 D plane or 3 D volume Credits: Sprengel et al, EMBS (1996) CSE 554 Extrinsic Deformations Slide 6

Thin-Plate Spline • A minimization problem – Minimizing distances between source and target points – Minimizing distortion of the space (as if bending a thin sheet of metal) • There is a closed-form solution – Solving a linear system of equations CSE 554 Extrinsic Deformations Slide 7

Thin-Plate Spline • Input pi qi – Source points: p 1, …, pn – Target points: q 1, …, qn • Output p – A deformation function f[p] for any point p f[p] CSE 554 Extrinsic Deformations Slide 8

Thin-Plate Spline • Minimization formulation – Ef: fitting term • Measures how close is the deformed source to the target – Ed: distortion term • Measures how much the space is warped – : weight • Controls how much non-rigid warping is allowed CSE 554 Extrinsic Deformations Slide 9

Thin-Plate Spline • Fitting term – Minimizing sum of squared distances between deformed source points and target points CSE 554 Extrinsic Deformations Slide 10

Thin-Plate Spline • Distortion term – Minimizing a physical bending energy on a metal sheet (2 D): – The energy is zero when the deformation is affine • Translation, rotation, scaling, shearing CSE 554 Extrinsic Deformations Slide 11

Thin-Plate Spline • Finding the minimizer for – Uniquely exists, and has a closed form: where • M: an affine transformation matrix • vi: translation vectors (one per source point) • Both M and vi are determined by pi, qi, CSE 554 Extrinsic Deformations Slide 12

Thin-Plate Spline • Result – At higher , the deformation is closer to an affine transformation Credits: Sprengel et al, EMBS (1996) CSE 554 Extrinsic Deformations Slide 13

Thin-Plate Spline • Application: landmark-based image registration – Manual or automatic detection of landmarks and correspondences Source Target Deformed source Credits: Rohr et al, TMI (2001) CSE 554 Extrinsic Deformations Slide 14

Free Form Deformation • Uses a control lattice that embeds the shape • Deforming the lattice points warps the embedded shape Credits: Sederberg and Parry, SIGGRAPH (1986) CSE 554 Extrinsic Deformations Slide 15

Free Form Deformation • Warping the space by “blending” the deformation at the control points – Each deformed point is a weighted sum of deformed lattice points CSE 554 Extrinsic Deformations Slide 16

Free Form Deformation • Input – Source lattice points: p 1, …, pn – Target lattice points: q 1, …, qn • Output – A deformation function f[p] for any p f[p] point p in the lattice grid. pi qi • wi[p]: pre-computed “influence” of pi on p CSE 554 Extrinsic Deformations Slide 17

Free Form Deformation • Desirable properties of the weights wi[p] – Greater when p is closer to pi • So that the influence of each control point is local – Smoothly varies with location of p • So that the deformation is smooth p f[p] – • So that f[p] = wi[p] qi is an affine combination of qi pi qi – • So that f[p]=p if the lattice stays unchanged CSE 554 Extrinsic Deformations Slide 18

Free Form Deformation • Finding weights (2 D) – Let the lattice points be pi, j for i=0, …, k and j=0, …, l – Compute p’s relative location in the grid (s, t) • Let (xmin, xmax), (ymin, ymax) be the range of grid p 0, 2 t p 1, 2 p 2, 2 p 3, 2 p p 0, 1 p 1, 1 p 2, 1 p 3, 1 p 0, 0 p 1, 0 p 2, 0 p 3, 0 s CSE 554 Extrinsic Deformations Slide 19

Free Form Deformation • Finding weights (2 D) – Let the lattice points be pi, j for i=0, …, k and j=0, …, l – Compute p’s relative location in the grid (s, t) – The weight wi, j for lattice point pi, j is: p 0, 2 t p 1, 2 p 2, 2 p 3, 2 p p 0, 1 p 1, 1 p 2, 1 p 3, 1 p 0, 0 p 1, 0 p 2, 0 p 3, 0 s • i, j: importance of pi, j • B: Bernstein basis function: CSE 554 Extrinsic Deformations Slide 20

Free Form Deformation • Finding weights (2 D) – Weight distribution for one control point (max at that control point): p 3, 2 p 3, 1 p 1, 1 p 0, 2 p 3, 0 p 0, 1 p 2, 0 p 1, 0 p 0, 0 CSE 554 Extrinsic Deformations Slide 21

Free Form Deformation • A deformation example CSE 554 Extrinsic Deformations Slide 22

Free Form Deformation • Image registration – Embed the source in a lattice – Compute new lattice positions over the target • Manually, or solve it as an optimization problem (maximally matching images contents while minimizing distortion) – Deform each source pixel using FFD www. slicer. org CSE 554 Extrinsic Deformations Slide 23

Cage-based Deformation • Use a control mesh (“cage”) to embed the shape • Deforming the cage vertices warps the embedded shape Credits: Ju, Schaefer, and Warren, SIGGRAPH (2005) CSE 554 Extrinsic Deformations Slide 24

Cage-based Deformation • Warping the space by “blending” the deformation at the cage vertices pi p – wi[p]: pre-computed “influence” of pi on p f[p] CSE 554 Extrinsic Deformations qi Slide 25

Cage-based Deformation • Finding weights (2 D) – Problem: given a closed polygon (cage) with vertices pi and an interior point p, find smooth weights wi[p] such that: • 1) pi • 2) CSE 554 p Extrinsic Deformations Slide 26

Cage-based Deformation • Finding weights (2 D) – A simple case: the cage is a triangle – The weights are unique (3 eqs, 3 vars) p 1 p – Known as the barycentric coordinates of p p 3 p 2 CSE 554 Extrinsic Deformations Slide 27

Cage-based Deformation • Finding weights (2 D) – The harder case: the cage is an arbitrary (possibly concave) polygon – The weights are not unique • A good choice: Mean Value Coordinates (MVC) • Can be extended to 3 D pi-1 pi αi p αi+1 pi+1 CSE 554 Extrinsic Deformations Slide 28

Cage-based Deformation • Finding weights (2 D) – Weight distribution of one cage vertex in MVC: pi CSE 554 Extrinsic Deformations Slide 29

Cage-based Deformation • Application: character animation CSE 554 Extrinsic Deformations Slide 30

Cage-based Deformation • Registration – Embed source in a cage – Compute new locations of cage vertices over the target • Minimizing some fitting and energy objectives – Deform source pixels using MVC • Not seen in literature yet… future work! CSE 554 Extrinsic Deformations Slide 31

Further Readings • Thin-plate spline deformation – “Principal warps: thin-plate splines and the decomposition of deformations”, by Bookstein (1989) – “Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines”, by Rohr et al. (2001) • Free form deformation – “Free-Form Deformation of Solid Geometric Models”, by Sederberg and Parry (1986) – “Extended Free-Form Deformation: A sculpturing Tool for 3 D Geometric Modeling”, by Coquillart (1990) • Cage-based deformation – “Mean value coordinates for closed triangular meshes”, by Ju et al. (2005) – “Harmonic coordinates for character animation”, by Joshi et al. (2007) – “Green coordinates”, by Lipman et al. (2008) CSE 554 Extrinsic Deformations Slide 32