The University of Ontario Snakes Shapes and Gradient

The University of Ontario Snakes, Shapes, and Gradient Vector Flow Ayesha Mehboob Memoona Iftikhar 5 -1

The University of Content Ontario What is Snake/Active Contour Model n Applications n Working of traditional Snake Model n Improvements to Traditional Snake n GVF Snake Model n Examples of GVF n 5 -2

The University of “Snakes” Ontario • Snakes, active contours [Kass, Witkin, Terzopoulos 1987] • In general, deformable models are widely used Given: initial contour (model) near desirable object 5 -3

The University of “Snakes” • Snakes, active contours Ontario [Kass, Witkin, Terzopoulos 1987] Given: initial contour (model) near desirable object Goal: evolve the contour to fit exact object boundary 5 -4

The University of gradient descent w. r. t. some function describing snake’s quality “Snakes” Ontario A smooth 2 D curve which matches to image data n Initialized near target, iteratively refined n Can restore missing data n initial intermediate final Q: How does that work? …. 5 -5

The University of Applications Ontario 5 -6

The University of Applications Ontario 5 -7

The University of Ontario WORKING OF TRADITIONAL SNAKE 5 -8

We have Preview The University of Ontario n 5 -9

The University of Preview f(x) for simplicity, assume that "snake” is a vector (or point) in R 1 Ontario assume some energy function f(x) describing snake’s “quality” local minima for f(x) gradient descent for 1 D functions Q: Is snake (contour) a point in some space? . . . Yes 5 -10

The University of Parametric Curve Representation (continuous case) n Ontario A curve can be represented by 2 functions parameter open curve closed curve Note: in computer vision and medical imaging the term “snake” is commonly associated with such parametric representation of contours. (Other representations will be discussed later!) 5 -11

The University of Parametric Curve Representation (discrete case) n Ontario A curve can be represented by a set of 2 D points parameter 5 -12

Measuring snake’s quality: Energy function The University of Ontario We can define some energy function E(C) that assigns some number (quality measure) to all possible snakes WHY? : Somewhat philosophical question, but E(C) (contours C) specifying a quality function E(C) is an objective (scalars) way to define what “good” means for contours C. Moreover, one can find “the best” contour (segmentation) by optimizing energy E(C). 5 -13

The University of Energy function Ontario Usually, the total energy of snake is a combination of internal and external energies Internal energy encourages smoothness or any particular shape Internal energy incorporates prior knowledge about object boundary allowing to extract boundary even if some image data is missing External energy encourages curve onto image structures (e. g. image edges) 5 -14

The University of Internal Energy (continuous case) n Ontario The smoothness energy at contour point v(s) could be evaluated as Elasticity/stretching Stiffness/bending Then, the interior energy (smoothness) of the whole snake is 5 -15

The University of Internal Energy (discrete case) Ontario elastic energy (elasticity) bending energy (stiffness)

The University of Internal Energy (discrete case) Ontario Elasticity Stiffness 5 -17

The University of External energy Ontario The external energy describes how well the curve matches the image data locally n Numerous forms can be used, attracting the curve toward different image features n 5 -18

The University of External energy Ontario n Suppose we have an image I(x, y) Can compute image gradient at any point Edge strength at pixel (x, y) is External energy of a contour point v=(x, y) could be n External energy term for the whole snake is n n n continuous case discrete case 5 -19

The University of Basic Elastic Snake n Ontario The total energy of a basic elastic snake is continuous case discrete case elastic smoothness term (interior energy) image data term (exterior energy) 5 -20

The University of Basic Elastic Snake (discrete case) Ontario This can make a curve shrink (to a point) i-1 i Li-1 Li i+1 Li+1 i+2 C 5 -21

The University of Basic Elastic Snake (discrete case) n The problem is to find contour that minimizes n Optimization problem for function Ontario • can compute local minima via gradient descent 5 -22

The University of Basic Elastic Snake Synthetic example Ontario (1) (2) (3) (4) 5 -23

Basic Elastic Snake Dealing with missing data n The University of Ontario The smoothness constraint can deal with missing data: 5 -24

The University of Basic Elastic Snake Relative weighting Ontario n Notice that the strength of the internal elastic component can be controlled by a parameter, n Increasing this increases stiffness of curve large medium small 5 -25

The University of Discrete Snakes Optimization n Ontario At each iteration we compute a new snake position within proximity to the previous snake New snake energy should be smaller than the previous one Stop when the energy can not be decreased within local neighborhood of the snake (local energy minima) Optimization Methods • Gradient Descent 5 -26

The University of Gradient Descent n Ontario Example: minimization of functions of 2 variables negative gradient at point (x, y) gives direction of the steepest descent towards lower values of function E 5 -27

The University of Gradient Descent n Ontario Example: minimization of functions of 2 variables update equation for a point p=(x, y) Stop at a local minima where 5 -28

The University of Gradient Descent for Snakes Ontario simple elastic snake energy C here, energy is a function of 2 n variables update equation for the whole snake C 5 -29

The University of Gradient Descent for Snakes Ontario simple elastic snake energy C here, energy is a function of 2 n variables update equation for each node C 5 -30

The University of Problems with snakes n Ontario External energy: may need to diffuse image gradients, otherwise the snake does not really “see” object boundaries in the image unless it gets very close to it. image gradients are large only directly on the boundary 5 -31

The University of Problems with snakes Ontario (a) Convergence of a snake using (b) traditional potential forces, and (c) shown closeup within the boundary concavity. 5 -32

The University of Ontario IMPROVEMENTS IN TRADITIONAL SNAKE 5 -33

The University of Alternative Way to Improve External Energy n Use Ontario instead of • Distance Transform where D() is (for detected binary image features, e. g. edges) binary image features (edges) Distance Transform can be visualized as a grayscale image Distance Transform • Generalized Distance Transform (directly for image gradients) 5 -34

The University of Distance Transform (see p. 20 -21 of the text book) Ontario Image features (2 D) 1 1 2 3 4 5 Distance Transform 0 1 2 3 4 3 0 1 2 3 3 2 0 1 2 3 2 1 0 0 1 2 1 0 1 2 2 2 1 0 1 3 3 2 1 0 1 4 4 3 2 1 0 1 2 2 2 1 Distance Transform is a function that for each image pixel p assigns a non-negative number corresponding to distance from p to the nearest feature in the image I 5 -35

The University of Problem with distance Ontario (a) Convergence of a snake using (b) distance potential forces, and (c) shown close-up within the boundary concavity. 5 -36

The University of Ontario Gradient Vector Flow (GVF) (A new external force for snakes) • Detects shapes with boundary concavities. • Large capture range.

The University of Diffusing Image Gradients Ontario image gradients diffused via Gradient Vector Flow (GVF) Chenyang Xu and Jerry Prince, 98 http: //iacl. ece. jhu. edu/projects/gvf/ 5 -38

The University of Model for GVF snake Ontario The GVF field is defined to be a vector field V(x, y) = Force equation of GVF snake n n n V(x, y) is defined such that it minimizes the energy functional f(x, y) is the edge map of the image.

The University of Ontario n | f | is small, energy dominated by first term n n • ( smoothing ) | f | is large, second term dominates • minimal when v= f μ is tradeoff parameter, increase with noise

The University of Diffusion Equations Ontario n GVF field can be obtained by solving following Euler equations 2 Is the Laplacian operator. n n Reason for detecting boundary concavities. The above equations are solved iteratively using time derivative of u and v.

The University of Diffussion Equation n Ontario The steady-state solution of these linear parabolic equations is the desired solution 5 -42

The University of Ontario n For convenience, we rewrite (14) as follows: 5 -43

The University of Ontario n To set up the iterative solution, let the indices i, j, and n correspond to x, y , and t, respectively, 5 -44

The University of Ontario n Substituting these approximations into (15) gives our iterative solution to GVF as follows: 5 -45

The University of Stability of GVF Ontario n 5 -46

The University of Results of GVF Field Ontario (a) Convergence of a snake using (b) GVF external forces, and (c) shown close-up within the boundary concavity. 5 -47

The University of Traditional external force field v/s GVF field Ontario Traditional force GVF force (Diagrams courtesy “Snakes, shapes, gradient vector flow”, Xu, Prince)

The University of Difference in Results Ontario Image with initial contour GVF snake Traditional snake

The University of Snake Initialization and Convergence Ontario (a) Initial curve and snake results from (b) a balloon with an outward pressure, (c) a distance potential force snake, and (d) a GVF snake. 5 -50

The University of Snake Initialization and Convergence Ontario (a) Initial curve and snake results from (b) a traditional snake, (c) a distance potential force snake, and (d) a GVF snake. 5 -51

The University of References Ontario n M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models. “, International Journal of Computer Vision. v. 1, n. 4, pp. 321 -331, 1987. n Laurent D. Cohen , “Note On Active Contour Models and Balloons“, CVGIP: Image Understanding, Vol 53, No. 2, pp 211 -218, Mar. 1991. n C. Xu and J. L. Prince, “Gradient Vector Flow: A New External Force for Snakes”, Proc. IEEE Conf. on Comp. Vis. Patt. Recog. (CVPR), Los Alamitos: Comp. Soc. Press, pp. 66 -71, June 1997.