Dense optical flow estimation in image sequences and

Dense optical flow estimation in image sequences and disparity map computation for stereo pairs applied to 3 D reconstruction Javier Sánchez Pérez Universidad de Las Palmas de Gran Canaria Departamento de Informática y Sistemas Grupo de Análisis Matemático de Imágenes

Contents • Optical flow methods – Standard approach – Considering colour images – Making the method symmetric • Stereoscopic methods – – Epipolar geometry Standard approach Colour method Symmetrical method • Main contributions and conclusions

Optical flow Standard approach • • • Variational approach Energy minimization Large displacements Numerical scheme Experimental results

Variational approach • Energy

Regularizing term • Nagel-Enkelmann operator

Energy minimization • Associated Euler-Lagrange equations

Energy minimization • Gradient descent • Anisotropic diffusion – diffusion tensor

Large displacements • Pyramidal approach

Large displacements

Large displacements • Scale-Space approach

Large displacements

Invariance • Invariance under gray level shifts

Numerical scheme

Experimental results • Squares sequence

Experimental results

Experimental results • Square sequence

Experimental results Comparison with other methods in Barron et al. 100% dense methods Method Horn y Schunk Ang. Error 32, 81º Stand. Dev. 13, 67º Nagel Anandan Singh Ours 34, 57º 31, 46º 46, 12º 10, 97º 14, 38º 18, 31º 18, 64º 9, 60º

Experimental results • Taxi sequence

Experimental results • Yosemite sequence

Experimental results Comparison with other methods in Barron et al. 100% dense methods Method Horn y Schunk Ang. Error 9, 78º Stand. Dev. 16, 19º Nagel Anandan Singh Ours 10, 22º 13, 36º 10, 03º 5, 53º 16, 51º 15, 64º 13, 13º 7, 40º

Optical flow for color images • Energy

Results • Cube sequence

Symmetrical optic flow • Composed energy

Variational approach • Energies

Energy minimization • Choosing the function • Iterative scheme

Experimental results • Squares sequence Occlusions Nonsymmetric Symmetric Euclid. error Ang. error 0. 16 0. 50º 0. 081 0. 18º

Experimental results • Marble blocks (H. H. Nagel) Nonsymmetric Symmetric Euclid. error 0. 37 Ang. error 9. 4º 0. 17 5. 2º

Stereoscopic vision • • Epipolar geometry Variational approach Energy minimization Experimental results

Epipolar geometry Fundamental matrix Epipolar lines m t 2 F m 1=0 I 2 I 1 m 2 C 1 M

Variational approach • Embedding the epipolar geometry I 2 I 1 m 1 m 2

Variational approach • Energy

Experimental results • Hervé’s face

Experimental results • Inria Library

Stereo for color images

Experimental results

Experimental results

Symmetrical stereo method • Energy

Experimental results

Main contributions • Common framework for optical flow and stereoscopic vision • Improvement of Nagel-Enkelmann method – Consistent centralization of both terms – Grey level shift invariance • Scale space approach to estimate large displacements • Extensions to colour images • Symmetrization of results

Conclusions • Good accuracy for large displacements • Improvement of dense methods studied in the work by Barron et al. • High accuracy of 3 D reconstructions • Coherence in both directions • Natural way of detecting occlusions