Active Contours Planes Sebastian Thrun Gary Bradski Daniel

Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS 223 B Computer Vision http: //robots. stanford. edu/cs 223 b Some slides taken from: Julien Jomier

Outline Snakes n Planes n Sebastian Thrun Stanford University CS 223 B Computer Vision

Introduction (1) n The active contour model, or snake, is defined as an energy-minimizing spline. n Active contours results from work of Kass et. al. in 1987. n Active contour models may be used in image segmentation and understanding. Sebastian Thrun Stanford University CS 223 B Computer Vision

Example Sebastian Thrun Stanford University CS 223 B Computer Vision

Introduction (2) Aorta segmentation using active contours Sebastian Thrun Stanford University CS 223 B Computer Vision

Introduction (3) First an initial spline (snake) is placed on the image, and then its energy is minimized. n Local minima of this energy correspond to desired image properties. n Unlike most other image models, the snake is active, always minimizing its energy functional, therefore exhibiting dynamic behavior. n Also suitable for analysis of n Sebastian Thrun Stanford University CS 223 B Computer Vision

Kass Algorithm n The snake is defined parametrically as v(s)=[x(s), y(s)], where s [0, 1] is the normalized arc length along the contour. The energy functional to be minimized may be written as Econt = snake continuity ESebastian snake curvature Stanford University curv = Thrun CS 223 B Computer Vision

Internal Energy n Continuity: n Curvature: Sebastian Thrun Stanford University CS 223 B Computer Vision

Image Forces n Dark/Bright Lines n Edges Sebastian Thrun Stanford University CS 223 B Computer Vision

Trade-offs n a, b, g determine trade-off Sebastian Thrun Stanford University CS 223 B Computer Vision

Numerical Algorithm n n Select N initial locations p 1, …, p. N Update until convergence Sebastian Thrun Stanford University CS 223 B Computer Vision

Snakes, Done Right n Define Spline over p 1, …, p. N Optimize criterion for all points on spline Allow for corners n (optimization becomes tricky, fills entire literature) n n Sebastian Thrun Stanford University CS 223 B Computer Vision

Applications http: //www. markschulze. n et/snakes/ Main Applications are: n Segmentation n Tracking n Registration Sebastian Thrun Stanford University CS 223 B Computer Vision

Examples (1) Julien Jomier Sebastian Thrun Stanford University CS 223 B Computer Vision

Examples (2) Julien Jomier Sebastian Thrun Stanford University CS 223 B Computer Vision

Examples (3) Heart Sebastian Thrun Stanford University Julien Jomier CS 223 B Computer Vision

Examples (4) 3 D Segmentation of the Hippocampus Julien Jomier Sebastian Thrun Stanford University CS 223 B Computer Vision

Examples (5) Julien Jomier Sebastian Thrun Stanford University CS 223 B Computer Vision

Example (6) Julien Jomier Sebastian Thrun Stanford University CS 223 B Computer Vision

Example (7) Julien Jomier Sebastian Thrun Stanford University CS 223 B Computer Vision

Problems with snakes n Snakes sometimes degenerate in shape by shrinking and flattening. n Stability and convergence of the contour deformation process unpredictable. Solution: Add some constraints n Initialization is not straightforward. Sebastian Thrun Stanford University CS 223 B Computer Vision

References n M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. In Proc. 1 st ICCV, pages 259 -268, June 1987. London, UK. n Yongjik Kim. A summary of Implicit Snake Formulation. n Jorgen Ahlberg. Active Contours in Three Dimensions. n M. Bertalmio, G. Sapiro and G. Randall. Morphing Active Contours. IEEE PAMI, Vol 22, Sebastian Thrun Stanford University CS 223 B Computer Vision

Outline Snakes n Planes n Sebastian Thrun Stanford University CS 223 B Computer Vision

Finding Planes n n n Lines in Image (e. g. , Hough transform) Planes in 3 D space (e. g. , stereo reconstruction, SFM) Problems: – – Number of planes Parameters of planes Data association: which point belongs to which plane Outlier removal (noise, non-flat surfaces) Sebastian Thrun Stanford University CS 223 B Computer Vision

Range Data in Multi-Planar Environment Taken with Laser Range Finder, but similar for Sebastian Thrun Stanford University CS 223 B Computer Vision

Basic Idea * Sebastian Thrun Stanford University * CS 223 B Computer Vision

Mathematical Model: Expectation Maximization n 3 D Model: Planar surface in 3 D y surface normal a surface z Distance point-surface displacement b x Sebastian Thrun Stanford University CS 223 B Computer Vision

Mixture Measurement Model n Case 1: Measurement zi caused by plane qj § Case 2: Measurement zi caused by something else Sebastian Thrun Stanford University CS 223 B Computer Vision

Measurement Model with Correspondences } correspondence variables C: Sebastian Thrun Stanford University CS 223 B Computer Vision

Expected Log-Likelihood Function …after some simple math probabilistic data association Sebastian Thrun Stanford University mapping with known data association CS 223 B Computer Vision

E-Step n Calculate expectations with fixed model q : (normalize so that Sebastian Thrun Stanford University ) CS 223 B Computer Vision

M-Step n Maximize subject to n Is equivalent to minimizing subject to Sebastian Thrun Stanford University CS 223 B Computer Vision

M-Step: Solve via Lagrange Multipliers n Define n And observe that Sebastian Thrun Stanford University CS 223 B Computer Vision

M-Step: Solve via Lagrange Multipliers n Solve for b : n Substitute back: n Is of the linear form: n Solution: two Eigenvectors of A with smallest Eigenvalues. Sebastian Thrun Stanford University CS 223 B Computer Vision

Determining Number of Surfaces Add Firstmodel Prune E/M M-Step M-step E-Steps components model component * * Sebastian Thrun J =2 =1 =3 * Stanford University CS 223 B Computer Vision

Choosing the “Right” Number of Planes: AIC J=0 J=1 J=2 J=3 J=4 J=5 increased data likelihood increased prior probability Sebastian Thrun Stanford University CS 223 B Computer Vision

Model Selection Approximately every 20 iterations of EM: n Start new surfaces – Near any set of collinear measurements n Terminate unsupported surfaces – If not supported by enough measurements – If density of measurements too low – If two planes are too close to each other Sebastian Thrun Stanford University CS 223 B Computer Vision

Results Sebastian Thrun Stanford University CS 223 B Computer Vision

Online Robotic Mapping @ CMU Sebastian Thrun Stanford University CS 223 B Computer Vision

Online Robotic Mapping @ Stanford Sebastian Thrun Stanford University CS 223 B Computer Vision