DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3

DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3 D RECONSTRUCTION USING COMPUTER VISION TECHNIQUES Teresa C. S. Azevedo João Manuel R. S. Tavares Mário A. P. Vaz

Contents I. Introduction to Computer Vision; II. Computer Platform presentation; III. Experimental results; IV. Conclusions; V. Future work. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 2

Computer Vision Introduction Platform Results Conclusions Future Work w Computer Vision is continuously trying to develop theories and methods for automatic extraction of useful information from images, as similar as possible to the complex human visual system. w Some applications: w Medicine - 3 D reconstruction / modelling, surgery planning; w Identification and navigation systems; w Virtual reality; w… Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 3

Goals and Methodology Introduction Platform Results Conclusions Future Work w Contactless techniques to recover the 3 D geometry of an object are usually divided in two classes: • active techniques - require some kind of energy projection or the camera’s (or object’s) movement to obtain 3 D information about the shape; • passive techniques - only use ambient light and so, usually, the extraction of 3 D information becomes more difficult. w Our goal was to obtain 3 D models of objects using an active vision technique called Structure From Motion (SFM). Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 4

Computer Platform Introduction Integration of functions for 3 D reconstruction, available from five software programs and one computational library, all open source: § • Open. CV; Ported to C using • Peter’s Matlab Functions; MATLAB Compiler toolbox • Torr’s Matlab Toolkit; • KLT; • Projective Rectification without Epipolar Geometry; • Depth Discontinuities by Pixel-to-Pixel Stereo. Platform Results Conclusions Future Work Modular structure; § User’s graphical interface; ++; § Computer language: C § § Developing tool: Microsoft Visual Studio, using MFC libraries (Microsoft Foundation Classes); § Operational system: Microsoft Windows. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 5

Computer Platform Introduction Platform Results w The functions integrated enclose several Computer Vision techniques: • feature points detection; • feature points matching between two images; • epipolar geometry Conclusions determination; Future Work • rectification; • dense matching. w For each technique, the user can easily choose the algorithm to use, as well as conveniently define its parameters. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 6

Feature Points detection Introduction Platform available algorithms for feature points detection Results Conclusions Future Work w Reflect the relevant discrepancies between their intensity values and those of their neighbours; w Usually represent vertices of objects, and their detection allows posterior matching between the images of the sequences. Open. CV Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz KLT 7

Feature Points matching Introduction Platform Results Conclusions Future Work available algorithms for feature points matching 1 st image feature points coordinates matching points coordinates on 2 nd image fundamental matrix w Image 2 D points association between sequential images, which are the projection of the same 3 D object point; w A short set of matching points is enough to determine the epipolar geometry between two images (the fundamental matrix). Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 8

Feature Points matching w Some results: Introduction Platform Results Conclusions Future Work Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 9

Epipolar Geometry determination Introduction Platform algorithms for epipolar geometry determination Results Conclusions Future Work algorithms for epipolar lines determination w Corresponds to the geometrical structure between two stereo images and its expressed mathematically by the fundamental matrix; w Also allows the elimination of some previous wrong matches (outliers), as well as make easier the determination of new matching points (dense matching). Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 10

Epipolar Geometry determination Introduction w Some results: Platform Results Conclusions Future Work Epipolar line Inlier Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 11

Rectification Introduction Platform Results Conclusions Future Work available algorithm for rectification w Method that changes two stereo images, in order to make them coplanar; w Performing this step makes dense matching easier to obtain; w The quality of the results is proportional to the quality of the epipolar geometry determination. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 12

Dense matching Introduction available algorithms for dense matching Platform Results Conclusions Future Work w Disparity map - codifies the distance between the object and the camera(s): closer points will have maximal disparity and farther points will get minimum disparity; w A disparity map gives some perception of discontinuity in terms of depth; w One of the algorithms also returns a discontinuity map – defines the pixels who border the changing between at least two levels of disparity. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 13

Dense matching Introduction w Some results: Original images Platform Results Conclusions Future Work Disparity map Discontinuity map Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 14

Conclusions Introduction Platform Results w The functions, already integrated in the computer platform, give good results when applied to objects with strong characteristics; Conclusions w From the experimental results, it is possible to conclude that low Future Work quality results are strongly correlated with few (strong) feature points detection and wrong matching; w This weakness is higher as the object shape variation is smooth. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 15

Future work Introduction Platform w The next steps of this work will focus on improving the results obtained when the objects have smooth and continuous surfaces: Results • inclusion of space carving techniques for object reconstruction; • the feature points to use in the 3 D space object definition will be detected Conclusions with the use of a reduced number of markers added on the object; Future Work estimation algorithms; • inclusion of a camera calibration technique, as well as pose and motion w Finally, the computer platform will be used in 3 D reconstruction and characterization of 3 D external human shapes. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 16

DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3 D RECONSTRUCTION USING COMPUTER VISION TECHNIQUES Acknowledgments This work was partially done in the scope of the project “Segmentation, Tracking and Motion Analysis of Deformable (2 D/3 D) Objects using Physical Principles”, reference POSC/EEA-SRI/55386/2004, financially supported by FCT - Fundação para a Ciência e a Tecnologia in Portugal. Teresa C. S. Azevedo João Manuel R. S. Tavares Mário A. P. Vaz