Multiview Stereo via Volumetric Graphcuts George Vogiatzis Philip

Dense Stereo reconstruction problem: • Input • Set of images of a scene I={I

Shape parametrisation • Disparity-map parameterisation • MRF formulation – good optimisation techniques exist (Graph-cuts,

Shape parametrisation • Volumetric parameterisation – e. g. Levelsets, Space carving etc. • Able

Solution ? • Cast volumetric methods in MRF framework • Benefits: • General surfaces

Graph cuts 24 5 23 40 13 30 21 20 12 13 5 1

Volumetric Graph cuts for segmentation • • Volume is discretized A binary MRF is

Volumetric Graph cuts for stereo • How to define ‘Inside’ and ‘Outside’ labels •

Volumetric Graph cuts Sink Source Min cut

Protrusion problem • ‘Balooning’ force • favouring bigger volumes L. D. Cohen and I.

Summary • Novel formulation for multiview stereo • Volumetric scene representation • Computationally tractable

Benefits 1. General surfaces and objects can be fully represented and computed as a

Slides: 29

Download presentation

Multi-view Stereo via Volumetric Graph-cuts George Vogiatzis, Philip H. S. Torr Roberto Cipolla

Shape From Images

Dense Stereo reconstruction problem: • Input • Set of images of a scene I={I 1, …, IK} • Camera matrices P 1, …, PK • Output • Surface model

Shape parametrisation • Disparity-map parameterisation • MRF formulation – good optimisation techniques exist (Graph-cuts, Loopy BP) • MRF smoothness is viewpoint dependent • Disparity is unique per pixel – only functions represented

Shape parametrisation • Volumetric parameterisation – e. g. Levelsets, Space carving etc. • Able to cope with non-functions • Convergence properties not well understood • Memory intensive • For Space carving, no simple way to impose surface smoothness

Solution ? • Cast volumetric methods in MRF framework • Benefits: • General surfaces can be represented • Optimisation is tractable (MRF solvers) • Occlusions can be approximately modelled • Smoothness is viewpoint independent

Graph cuts 24 5 23 40 13 30 21 20 12 13 5 1 4 3 50 40

Graph cuts 24 5 23 40 13 30 21 20 12 13 5 1 4 3 5+5+1+4+3=18 50 40

Volumetric Graph cuts for segmentation • • Volume is discretized A binary MRF is defined on the voxels Regular grid (6 or 26 neighbourhood) Voxels are labelled as OBJECT and BACKGROUND • Labelling cost set by OBJECT / BACKGROUND intensity statistics • Compatibility cost set by edge intensities

Volumetric Graph cuts for stereo • How to define ‘Inside’ and ‘Outside’ labels • How to deal with occlusion

Volumetric Graph cuts Sink Source Min cut

Face

Face - Visual Hull

Face - Slice

Face - Slice with graphcut

Face - Reconstruction

Protrusion problem • ‘Balooning’ force • favouring bigger volumes L. D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2 -d and 3 -d images. PAMI, 15(11): 1131– 1147, November 1993.

Protrusion problem

Graph

Results • Model House

Results • Model House – Visual Hull

Results • Model House

Results • Stone carving

Results • Haniwa

Summary • Novel formulation for multiview stereo • Volumetric scene representation • Computationally tractable global optimisation using Graph-cuts. • Visual hull for occlusions and geometric constraint

Benefits 1. General surfaces and objects can be fully represented and computed as a single surface. 2. The representation and smoothness constraint is image and viewpoint independent. 3. Multiple views of the scene can be used with occlusions approximately modelled. 4. Optimisation is computationally tractable, using existing max-flow algorithms.

Related presentations