Medial Object Shape Representations for Image Analysis Object
Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University of North Carolina, USA Credits: Many on MIDAG, especially Daniel Fritsch, Andrew Thall, George Stetten, Paul Yushkevich MIDAG@UNC
Medial Object Shape Representations for Image Analysis & Object Synthesis MIDAG@UNC
What shape representation is for ä Analysis from images ä Extract the kidney-shaped object ä Register based on the pelvic bone shapes ä Extract shape information w/o model ä Synthesis ä Design the object ä Deform the object, with physical realism ä Shape science ä Shape and biology ä Shape-based diagnosis MIDAG@UNC
What shape representation is for ä Analysis from images ä Extract the kidney-shaped object ä Register based on the pelvic bone shapes MIDAG@UNC
What shape representation is for ä Synthesis ä Design the object ä Deform the object, with physical realism MIDAG@UNC
What shape representation is for ä Shape science ä Shape and biology ä Shape-based diagnosis Brain structures (Gerig) MIDAG@UNC
Shape Sciences ä Geometry ä The spatial layout: via primitives Landmarks ä Boundary places and orientations ä Medial places, figural sizes and orientations ä Space itself ä ä Statistics ä The average shape ä Modes of variation from the average ä Computer Graphics ä Image Analysis MIDAG@UNC
Options for Primitives ä Space: xi for grid elements ä Landmarks: xi described by local geometry ä Boundary: (xi , normali) spaced along boundary ä Figural: nets of diatoms sampling figures MIDAG@UNC
Primitives for shape representation: Landmarks ä Sets of points of special geometry MIDAG@UNC
Primitives for shape representation: Boundaries ä Boundary points with normals MIDAG@UNC
Object Representation by M-Reps MIDAG@UNC
Each M-figure Represented by Net of Medial Primitives MIDAG@UNC
Each M-figure Represented by Net of Medial Primitives MIDAG@UNC
Figural Models ä Figures: successive medial involution ä ä ä Main figure Protrusions Indentations Separate figures Hierarchy of figures ä ä ä Relative position Relative width Relative orientation MIDAG@UNC
Primitives’ Desired Properties ä Geometry ä Intuitive: simple, global + local ä Efficiently deformable ä Easily extracted or created ä Spatial tolerance inherent ä Statistics ä Unimodality: normally distributed ä Via geometrical, tolerance-sensitive metric MIDAG@UNC
Figural Models with Boundary Deviations ä Hypothesis ä At a global level, a figural model is the most intuitive ä At a local level, boundary deviations are most intuitive MIDAG@UNC
Union and Difference of M-figures MIDAG@UNC
Medial Primitives ä x, (b, n) frame, r, q (object angle) ä Imply boundary segments with tolerance n ä Similarity transform equivariant ä Zoom invariance implies width-proportionality of ä tolerance of implied boundary ä boundary curvature distribution ä spacing along net ä interrogation aperture for image MIDAG@UNC
3 D kidney model extracted from CT Four figure model of the kidneys Red represents indentation figures MIDAG@UNC
Need for Special End Primitives ä Represent ä non-blobby objects ä angulated edges, corners, creases ä still allow rounded edges , corners, creases ä allow bent edges ä But ä Avoid infinitely fine medial sampling ä Maintain tangency, symmetry principles MIDAG@UNC
End Primitives Corner primitive in cross-section Rounded end primitive in cross-section MIDAG@UNC
Displacements from Figurally Implied Boundary implied by figural model Boundary after displacements MIDAG@UNC
Coarse-to-fine representation ä For each of three levels ä Figural hierarchy ä For each figure, net chain, successively smaller tolerance ä For each net tile, boundary displacement chain MIDAG@UNC
Multiscale Medial Model ä From larger scale medial net ä Coarsely sampled ä Smooother figurally implied boundary ä Larger tolerance ä Interpolate smaller scale medial net ä Finer sampled ä More detail in figurally implied boundary ä Smaller tolerance ä Represent medial displacements MIDAG@UNC
Multiscale Medial Model ä From larger scale medial net, interpolate smaller scale medial net and represent medial displacements b. MIDAG@UNC
Multiscale Medial/Boundary Model ä From medial net ä Coarsely sampled, smoother implied boundary ä Larger tolerance ä Represent boundary displacements along implied normals ä Finer sampled, more detail in boundary ä Smaller tolerance MIDAG@UNC
Shape Rep’n in Image Analysis ä Segmentation ä Extract an object from image ä Registration ä Find geometric transformation that brings two images into alignment ä 3 D/3 D ä 3 D/2 D ä Shape Measurement ä Find how probable a shape is MIDAG@UNC
Shape Repres’n in Image Analysis ä Segmentation äFind the most probable deformed mean model, given the image äProbability involves ä Probability of the deformed model (prior) ä Probability of the image, given the deformed model (likelihood) MIDAG@UNC
Probability of a deformed model ä From training set ä via principal components analysis, coarse-to-fine ä -C * Geometric difference from typical shape MIDAG@UNC
Medialness: medial strength of a medial primitive in an image ä Probability of image | deformed model ä Sum of boundariness values ä at implied boundary positions ä in implied normal directions ä with apertures proportional to tolerance ä Boundariness value ä Intensity profile distance from mean (at scale) ä ä ä statistical, based on training set Intensity differences via Gaussian derivatives MIDAG@UNC
Figurally implied boundaries and rendering, via 4 -figure model MIDAG@UNC
3 D DSL Model Deformation Initial Position of Model in Target Image MIDAG@UNC
3 D DSL Model Deformation Figural Deformation Iteration 3 MIDAG@UNC
3 D DSL Model Deformation with interfigural penalties Initial position After optimization MIDAG@UNC
Shape Repres’n in Image Analysis ä Registration ä Find the most probable deformation, given the image MIDAG@UNC
Shape Rep’n in Image Analysis ä Prior-free medial shape analysis ä Cores: height ridges of medialness (Pizer, Fritsch, Morse, Furst) ä Statistical (Stetten) analysis of medial diatoms MIDAG@UNC
Shape Rep’n in Image Analysis ä Cores: height ridges of medialness MIDAG@UNC
MIP@UNC
Shape Rep’n in Image Analysis ä Statistical analysis of medial diatoms MIDAG@UNC
sphere slab cylinder MIDAG@UNC
sphere slab cylinder MIDAG@UNC
sphere slab cylinder MIDAG@UNC
MIDAG@UNC
sphere slab cylinder MIDAG@UNC
sphere slab cylinder MIDAG@UNC
MIDAG@UNC
Shape Rep’n in CAD/CAM ä Stock figural models ä Deformation tools: large scale ä Coarse-to-fine specification ä Figural connection tools ä Direct rendering, according to display needs MIDAG@UNC
Deformation in CAD/CAM MIDAG@UNC
Shape Rep’n in CAD/CAM ä Design models for image analysis MIDAG@UNC
Medial Object Shape Representations for Image Analysis & Object Synthesis ä Figural models, at successive levels of tolerance ä Boundary displacements ä Work in progress ä Segmentation and registration tools ä Statistical analysis of object populations ä CAD tools, incl. direct rendering ä Connection relative critical manifolds ä… MIDAG@UNC
Application: Image guided planning & delivery of radiotherapy ä Planning in 3 D ä Extracting normal anatomy ä Extracting tumor ä Planning beam poses ä Patient placement ä Verification of plan via portal image MIDAG@UNC
Finding Treatment Pose from Portal Radiograph and Planning DRR MIDAG@UNC
Medial Net Shape Models Medial nets, positions only MIDAG@UNC
Integrated Medialness vs. Pose Offset MIDAG@UNC
MIDAG@UNC
Representing Boundary Displacements ä Along figurally implied boundary normals ä Coarse-to-fine ä Captures along-boundary covariance ä Useful for rendering MIDAG@UNC
Summing Medialness on Medial Net via Medial Weighting Function MIDAG@UNC
CT Slice of Kidneys in Abdomen MIDAG@UNC
Object Shape Brain structures (Gerig) MIDAG@UNC
Geometric aspects : Transformations ä Euclidean: ä translation and rotation ä Similarity: ä translation, rotation, zoom ä Affine MIDAG@UNC
- Slides: 60