Underwater reconstruction using depth sensors Alexandru Dancu 1

Underwater reconstruction using depth sensors Alexandru Dancu 1 Mickaël Fourgeaud 1 Zlatko Franjcic 2, 1 Razmik Avetisyan 3 1 Chalmers University of Technology 2 Qualisys AB 3 Rostock University

Contents • • • Related Work Background Experiment Discussion Conclusion

Related Work: Seafloor visualization • advances in sonar technology, positioning capabilities, and computational power revolutionized seafloor imaging • 1970’s, multibeam sonars were deepwater (12 k. Hz) systems, 16 beams • now, more than 100 beams, the angular Multibeam sonar-derived backscatter data draped sector covered > 120 degrees limited by over 20 m pixel bathymetry off Oahu, Hawaii attenuation in deeper water • shading, and 3 -D rendering was used with digital bathymetric data (DTM's) to form natural looking and easily interpretable, yet quantitative, landscapes. Mayer, Larry A. , et al. "Interactive 3 -D Visualization: A tool for seafloor navigation, exploration and engineering. " OCEANS 2000 MTS/IEEE Conference and Exhibition. Vol. 2. IEEE, 2000.

Related Work: opti-acoustic fusion • Acoustic cameras produce an image by recording the reflected sound, once the scene is insonified by acoustic pulse(s). In a 3 -D sonar, e. g. , Echoscope [18], the back -scattered signals are collected by a 2 -D array of transducers, and the image is Circles depict matching points in optical (top) and formed from “beam signals” sonar (bottom) Negahdaripour, Shahriar, Hicham Sekkati, and Hamed Pirsiavash. "Opti-acoustic stereo imaging: On system calibration and 3 -D target reconstruction. " Image Processing, IEEE Transactions on 18. 6 (2009): 1203 -1214.

Related Work: Clear underwater vision • Computer vision algorithm which inverts the image formation process, to recover a good visibility image of the object • doubles the underwater visibility range • based on modelling the image formation • model is relevent here Raw image Recovered image Based Schechner, Yoav Y. , and Nir Karpel. "Clear underwater vision" Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. Vol. 1.

Related Work: Clear underwater vision Signal • Point spread function empirical constants Based Schechner, Yoav Y. , and Nir Karpel. "Clear underwater vision" Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. Vol. 1.

Related Work • Voxelhashing

Background Dinast sensors Plastic box

Background: depth sensors Kinect Sensor Components and Specifications http: //msdn. microsoft. com/enus/library/jj 131033. aspx ASUS Xtion on Hardware. info http: //us. hardware. info/productinfo/12700 1/asus-xtion-pro

Background: Kinect IR emitter Infrared image of a child playing a Kinect game http: //youtu. be/0 Yr 5_0 imcb. Y

Background: Kinect IR emitter Infrared image of a child playing a Kinect game http: //youtu. be/0 Yr 5_0 imcb. Y

Water absorption of infrared Water absorption of visible and near-infrared spectrum (blue curve); Kinect IR emitter wavelength superimposed (red line), Based on data from GM Hale, MR Querry, Optical constants of water in the 200 nm to 200 mm wavelength, Applied Optics 12, 3 (1973), 555 --563.

Water refraction Refraction occurs when waves travel from a medium with a given refractive index to a medium with another at an oblique angle. v 1, v 2 are phase velocities of wave n 1, n 2 are refractive indices of medium Objects look bent because of refraction http: //en. wikipedia. org/wiki/File: Pencil_in_a_bowl_of_water. svg http: //upload. wikimedia. org/wikipedia/commons/0/04/Refraction. jpg http: //upload. wikimedia. org/wikipedia/commons/a/ac/Angle_of_incidence_Refraction_example. jpg

Water refraction Refraction occurs when waves travel from a medium with a given refractive index to a medium with another at an oblique angle. v 1, v 2 are phase velocities of wave n 1, n 2 are refractive indices of medium Misleading picture, the projection of Y and X on water line is the same, that is why it appears higher Objects look bent because of refraction Contracted http: //en. wikipedia. org/wiki/File: Pencil_in_a_bowl_of_water. svg http: //upload. wikimedia. org/wikipedia/commons/0/04/Refraction. jpg http: //upload. wikimedia. org/wikipedia/commons/a/ac/Angle_of_incidence_Refraction_example. jpg viewer observes smaller distance to the pencil particles because of smaller angle in water medium appears closer

Water refraction Refraction occurs when waves travel from a medium with a given refractive index to a medium with another at an oblique angle. viewer observes From water to air smaller distance n 1= nwater = 1. 3 to the pencil n = nair = 1. 0 particles because 2 of smaller angle n 1 > n 2, v 1 < v 2, in water medium so θ 1 < θ 2 appears closer From air to water n 1= nair = 1. 0 n 2= nwater = 1. 3 n 1 < n 2, v 1 > v 2, so θ 1 > θ 2 the ray in the higherindex medium is closer to the normal. Refraction of light at the interface between two media http: //upload. wikimedia. org/wikipedia/c ommons/thumb/3/3 f/Snells_law 2. svg/5 70 px-Snells_law 2. svg

Sensor inside water

Sensor above water

Sensor above water, reconstruction

Water refraction, reconstruction

Water refraction, reconstruction Above water • Mesh deforms, flattens Underwater From water to air n 1= nwater = 1. 3 n 2= nair = 1. 0 n 1 > n 2, v 1 < v 2, so θ 1 < θ 2

Water refraction, reconstruction Refraction Above water • Mesh deforms, flattens • Slope of mesh changes • Assumption: – Refraction responsible for mesh deformation Underwater From water to air n 1= nwater = 1. 3 n 2= nair = 1. 0 n 1 > n 2, v 1 < v 2, so θ 1 < θ 2

Water absorption, reconstruction

Water absorption, reconstruction Absorption Above water Underwater Based on data from GM Hale, MR Querry, Optical constants of water in the 200 nm to 200 mm wavelength, Applied Optics 12, 3 (1973), 555 --563.

Water absorption, reconstruction Absorption • Above water Underwater Based on data from GM Hale, MR Querry, Optical constants of water in the 200 nm to 200 mm wavelength, Applied Optics 12, 3 (1973), 555 --563.

Heatmap of mesh difference Based on the refraction and absorption assumption we created an error heatmap http: //www. danielgm. net/cc

Discussion • Discussion – Kinect it works and the other sensors do not because of infrared emitter power? ? • Angle don’t mention it!! • Drawing with kinect - 70 cm box idea • DIVA sponsor slide!!!

Future work • Zlatko

Ulf’s recommendations • Kinect operation range = 0. 8 m - 3. 5 m • 20 min maximum, max. 2 questions, should be shorter and at point, 15 min presentation + 5 min questions • Talk while showing the video, pause, stretch the video, parts should be proportional to what is said. . Moving object should be fast. • Q 1 – can u get this from white papers? • Emitter intensity (openkinect) (white paper)