SIEMENS CORPORATE RESEARCH Computing Exact Discrete Minimal Surfaces

SIEMENS CORPORATE RESEARCH Computing Exact Discrete Minimal Surfaces: Extending and Solving the Shortest Path Problem in 3 D with Application to Segmentation 1 Leo Grady Department of Imaging and Visualization Siemens Corporate Research

SIEMENS CORPORATE RESEARCH Outline 2 • • • Introduction Extending the shortest path problem to 3 D Method for computation Results Conclusion

SIEMENS CORPORATE RESEARCH Introduction – Shortest path in 3 D 3 A shortest path is easy to find in 3 D, but it is no longer a boundary

Introduction – Minimal boundary segmentation 4 4 image 4 4 weighted graph SIEMENS CORPORATE RESEARCH abstraction 4 Minimal segmentation specified two ways Cutting edges/specifying normals Specifying boundary Graph Cuts Max-flow/Min-cut Intelligent scissors/Live wire Dijkstra’s algorithm

Introduction – Minimal boundary segmentation = Shortest path on dual graph SIEMENS CORPORATE RESEARCH Specifying boundary 5 Sometimes called “cracks” or “edgels”

Introduction – Minimal boundary segmentation Primal - Cuts Dual - Surfaces Max-flow/Min-cut ? ? ? SIEMENS CORPORATE RESEARCH 4 4 4 volume 6

Introduction – Minimal boundary segmentation Q: Why bother if you have max-flow/min-cut? A: Different inputs, different outputs Shortest path/min surface Input: Source (inside) and terminal (outside) Input: A piece of the boundary Output: Closed contour/surface Output: Open contour/surface Different algorithmic complexity SIEMENS CORPORATE RESEARCH Max-flow/min-cut 7 Both algorithms persist in 2 D, but max-flow/min-cut only option in 3 D

SIEMENS CORPORATE RESEARCH Outline 8 • • • Introduction Extending the shortest path problem to 3 D Method for computation Results Conclusion

Extending the Shortest Path Problem SIEMENS CORPORATE RESEARCH Input: Two points (0 D boundary) Output: Minimum 1 D path having that boundary 9 minimal 1 -dimensional object 0 -dimensional boundary Path is shortest relative to weighting (metric) How to extend problem to 3 D?

Extending the Shortest Path Problem Input: Closed contour (1 D boundary) Output: Minimum 2 D surface having that boundary SIEMENS CORPORATE RESEARCH minimal 2 -dimensional object 10 1 -dimensional boundary Surface is minimal relative to weighting (metric)

Extending the Shortest Path Problem SIEMENS CORPORATE RESEARCH 2 D intelligent scissors/live wire ubiquitous segmentation method 11

Extending the Shortest Path Problem SIEMENS CORPORATE RESEARCH Red – Initial (boundary) contours Yellow – Computed minimal surface contours 12

SIEMENS CORPORATE RESEARCH Outline 13 • • • Introduction Extending the shortest path problem to 3 D Method for computation Results Conclusion

Method How to compute? SIEMENS CORPORATE RESEARCH Use boundary operator 14 : 1 D 0 D to enforce constraint : 2 D 1 D

Method Minimize path/surface SIEMENS CORPORATE RESEARCH 3 D 15 2 D - Binary vector indicating presence/absence of face in surface - Binary vector indicating presence/absence of edge in path - Weights of each face in the complex - Weights of each edge in the graph Use boundary operator as constraint Subject to: - Vector indicating edges in boundary - Vector indicating nodes in boundary Since boundary operator is linear, representable by a matrix Subject to: - Edge-face incidence matrix - Node-edge incidence matrix

Method - Digression Continuum interpretation - I Use generalized Stokes Theorem: 0 D Fundamental Theorem of calculus: : 2 D 1 D “Standard” Stokes Theorem SIEMENS CORPORATE RESEARCH : 1 D 16 Careful! Instead of bivectors, formulated in “primal” space

Method - Digression Continuum interpretation - II SIEMENS CORPORATE RESEARCH 2 D 17 3 D - Vector field taking nonzeros along minimal path - Vector field taking nonzeros on the normals of the surface - Ambient vector field (e. g. , derived from image gradients) Use boundary as constraint Subject to: RHS consists of two delta functions at endpoints RHS consists of a unit closed contour

Method Minimal surface problem SIEMENS CORPORATE RESEARCH Subject to: 18 Integer programming problem – Bad! However: Sometimes we can apply linear programming to an integer programming problem and guarantee an integer solution.

Method Minimal surface problem SIEMENS CORPORATE RESEARCH Subject to: 19 If constraints are feasible, then For some integer For the matrix . Therefore, spanning the nullspace of So, we can rewrite the problem in terms of Joint work with Vladimir Kolmogorov

Method Original minimal surface problem SIEMENS CORPORATE RESEARCH Subject to: 20 New minimal surface problem Subject to: Guaranteed to give an integer solution for an integer iff is totally unimodular. A matrix is totally unimodular if every square submatrix has a determinant equal to one of the set

Method New minimal surface problem SIEMENS CORPORATE RESEARCH Subject to: 21 Big question: What is ? The volume-face boundary operator : 3 D 2 D Followed by the : 2 D 1 D, gives zero May also be stated as: “The boundary of the boundary is zero”

Method SIEMENS CORPORATE RESEARCH Dual 22 Volume-face incidence in dual lattice is node-edge incidence in primal lattice Primal

Method New minimal surface problem SIEMENS CORPORATE RESEARCH Subject to: 23 Since all node-edge incidence matrices are totally unimodular is guaranteed to be integer, and therefore is guaranteed to be integer.

Method Conclusion: SIEMENS CORPORATE RESEARCH Minimal surface problem 24 Subject to: Solvable using generic linear programming code! (and so is the other formulation) Subject to:

SIEMENS CORPORATE RESEARCH Outline 25 • • • Introduction Extending the shortest path problem to 3 D Method for computation Results Conclusion

SIEMENS CORPORATE RESEARCH Results - Correctness 26

Results – 3 D Segmentation SIEMENS CORPORATE RESEARCH Red – Initial (boundary) contours Yellow – Computed minimal surface contours 27

SIEMENS CORPORATE RESEARCH Outline 28 • • • Introduction Extending the shortest path problem to 3 D Method for computation Results Conclusion

SIEMENS CORPORATE RESEARCH Conclusion 1. Natural extension of shortest path given two points is minimal surface given a closed contour 29 2. Minimal surface problem solvable with generic linear programming code 3. There are, in fact, two integral linear programming problems that could be solved to achieve the solution

Acknowledgements More Information Marie-Pierre Jolly – Posing the problem SIEMENS CORPORATE RESEARCH Gareth Funka-Lea – Support and enthusiasm for the work 30 Yuri Boykov – Enthusiasm and encouragement of the topic Chenyang Xu – Extensive comments on the paper Vladimir Kolmogorov – Technical analysis of LP problem Writings and code My webpage: http: //cns. bu. edu/~lgrady Combinatorial minimal surface MATLAB code: http: //cns. bu. edu/~lgrady/minimal_surface_matlab_code. zip MATLAB toolbox for graph theoretic image processing at: http: //eslab. bu. edu/software/graphanalysis/

Uniqueness A minimal surface/path is not necessarily unique SIEMENS CORPORATE RESEARCH 3 D 31 2 D

Method Is the face-edge boundary operator totally unimodular? SIEMENS CORPORATE RESEARCH Yes, provided that the complex is orientable. Fortunately, the lattices used in vision are all orientable. 32