A NEW USE OF AUTOMATIC MORPHING TECHNIQUES TO

A NEW USE OF AUTOMATIC MORPHING TECHNIQUES: TO CORRECT DISTORTION OF ENDOSCOPIC SYSTEMS Authors: Tiago Bonini Borchartt Marcos Cordeiro d’Ornellas Aura Conci Alicia Becerra Romero Paulo Henrique de Aguiar

Summary Introduction Endoscopy Image Proposed Method Morphing and Warping Affine Transformations Open. GL and GPU’s Results Future Works Conclusion 2

Introduction Images obtained through endoscopes have some distortion due to lens or all system used; This distortion affects the diagnostic; This work proposes a method to correct these distortions by applying techniques of morphing and warping. 3

Endoscopic Image Real examination endoscopic image 4

Proposed Method Correct the distortion using Morphing Techniques; Use linear equations to transform parts of the image separately, dividing the image into a triangular mesh; Create the mesh from a pattern pre-defined; Apply the same equations in real images; Improve the processing performance using GPU – graphic processing unit. 5

Algorithm: Over view:

Morphing and Warping Techniques Morphing Techniques are used to produce special effects in animations, games and movies; Morphing is often used to depict one person turning into another; The Automatic Warping technique uses a triangular grid and affine transformations to modify an original triangle in another triangle in a target position. 7

Chess-pattern Fig. 1: Chess-pattern used to calibration Fig. 2: Chess-pattern acquired by endoscope 8

Verify The Distortion Fig. 3: Fig. 2 minus Fig. 1 pre-processed (75. 3% matching) 9

Triangular Mesh Fig. 4: Mesh generated in Chess-pattern 10 Fig. 5: Mesh generated in endoscope image

Affine Transformations Triangle from chess-pattern Triangle from endoscopy image The system calculates the affine transformations that changes the position of the vertices of a triangle in the vertices of another triangle using the equation: wi = M. vi + t Written in two dimensions as: 11

Affine Transformations (2) Developing the matrix operations for each triangular vertex, we arrive to the following equations: 12

Affine Transformations (3) Are calculated the values of M and t from each triangle pair source/target The system applies the transformation to all pixels belonging to the respective triangle in the distorted image; This process is repeated for all triangles of the mesh; The same equations can be used to correct images of real endoscopy. 13

Problems Solved If a triangle has its area expanded : Problem: Provide values for pixels that presents not corresponded in the destination; Solution: We increase four times the image size with a bilinear filter, before applying the transform; If a triangle has its area reduced : 14 Problem: The possibility of multiple pixels being aggregated in the same region; Solution: The new color values of each pixel are defined as an arithmetic mean between the all possible values.

Open. GL and GPU’s The system uses texture mapping to render the result image directly into the video card; The coordinates of the triangular mesh of Fig. 4 is used by system to draw triangles in which the texture is defined by mapping the distorted image using the mesh of Fig. 5 as texture coordinates; Acceleration of approximately 700 x the processing of an image; Average processing time reduced from 14737 milliseconds (14. 737 seconds) to 21. 2 milliseconds. 15

Results – Chess-pattern Fig. 6: Result of applying the method in Fig. 2 16

Results – Verify the Distortion Fig. 7: Fig. 6 minus Fig. 1 pre-processed (94. 6% matching) 17

Results – Comparative Images Fig. 8: Real endoscopic image 18 Fig. 9: Fig. 8 processed by correction method

Results – Comparative Images (2) Fig. 10: Real endoscopic image 19 Fig. 11: Fig. 10 processed by correction method

Future Works To improve the results using morphing techniques is necessary to use a non uniform mesh and also test the system using other patterns for calibration; As future work we can handle the extra brightness in the image caused by light source coupled in the camera and try to optimize the algorithm to apply it directly in real time during the screening process; Try other techniques such as analytical and processing techniques using polar coordinates. 20

Future Works (cont. ) �Try other images for calibration like this using polar coordinates:

Conclusion The main advantage of here presented technique of morphing and warping for image correction is that these methods accept the use of several triangular regions in an image and warp each region in a different way; This method provides greater freedom to handle the image; Texture mapping is more adequate for real time processing. 22

References Borchartt, T. , 2009. More results and the executable software. In: http: //www. ic. uff. br/~uffufsm/? p=software. Conci, A. , Azevedo, E. , & Leta, F. , 2008. Computac¸ ˜ao Grafica: volume 2. Campus/Elsevier. Hartley, R. & Kang, S. , 2007. Parameter-free radial distortion correction with center of distortion estimation. IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 29, n. 8, pp. 1309– 1321. Helferty, J. , Zhang, C. , Mc. Lennan, G. , & Higgins, W. , 2001. Videoendoscopic distortion correction and its application to virtual guidance of endoscopy. IEEE Transaction on Medical Imaging, vol. 20, n. 7, pp. 605– 617. Kannala, J. & Brandt, S. , 2006. A generic camera model and calibration method for conventional, wide-angle and fish-eye lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, n. 8, pp. 1335– 1340. Möller, T. & Haines, E. , 2002. Real-Time Redering. Editorial Sales, 2 edition. Stehle, T. , Truhn, D. , Aach, T. , Trautwein, C. , & Tischendorf, J. , 2007. Camera calibration for fish-eye lenses in endoscopy with an application to 3 d reconstrution. IEEE Transaction on Biomedical Imaging, pp. 1176– 1179. Thank you! 23