Synthetically Distorted Image Restoration Muharrem Mercimek 24 August

Synthetically Distorted Image Restoration Muharrem Mercimek 24 August 2009 1

Contents • • Motivations Theory of two state of the art Non-rigid methods – Demons – Lucas Kanade’s Optical flow • • Synthetic Deformation and Correction Experiments Future work: A new method for non-rigid registration 2

Motivations Examples of Atmospheric Turbulences *1 *2 1 -http: //phasing. org/2009/08/27/the-other-meaning-of-dimension-and-its-use-in-physics/ 2 - "why stars twinkle“ www. youtube. com 3

Registration based Restoration Algorithm Blocks 1 - Calculate the Temporal Median of the image seq. , Use this as the reference image 2 - Apply geometric deformation correction 3 - Calculate the Temporal median of the corrected image seq. Use this as the final output that presents the scene image. • To restore the artefacts like speckles, and any kind of geometric deformation and dancing image effects caused by atmospheric turbulence • To model Atmospheric turbulence is extremely difficult. deformation correction function. Non-rigid registration Gilles, J. , Dagobert, T. , and Franchis, C. “Atmospheric Turbulence Restoration by Diffeomorphic Image Registration and Blind Deconvolution”. In Proceedings of the 10 th international Conference on ACIVS, October 20 - 24, 2008. 4

Transformation Models • • • T transformation function, xf , yf, zf coordinates of reference image, p transformation parameters, xr, yr, zr coordinates of reference image. – – – Rigid Affine Projective Non-Rigid or Elastic General linear parametric transformation model for 2 D * Michal Irani, P. Anandan: Robust Multi-Sensor Image Alignment. ICCV 1998: 959 -966 5

Deformable (non-rigid) image registration • Deformable (non-rigid) image registration algorithms can be categorized into two classes: Feature-based or intensity-based. • Feature-based methods – Extract visual features (corners, textured areas) and track them over multiple frames • Sparse motion fields, but possibly robust tracking • Suitable especially when image motion is large (10 -s of pixels) • Image intensity-based -methods – Directly recover image motion from spatial-temporal image intensity variations • Motion vectors directly recovered without an intermediate feature extraction step • Dense motion fields, but more sensitive to appearance variations • Suitable for video and when image motion is small (< 10 pixels) 6

Demons • • Base version Thirion’ s algorithm The original ‘demons’ algorithm used gradient information from a static reference image to determine the ‘demons’ force required to deform the ‘moving’ image. • Conceptually, the diffusing model assumes that local ‘demons’ at every voxel location are applying invisible ‘forces’ that push the voxels of the moving image into matching up with the reference (static) image. Thirion J P Non-rigid matching using demons Proc. CVPR ’ 96, 1996 IEEE Computer Society Conf. Computer Vision and Pattern Recognition pp 245– 51, 1996 7

Demons Estimated Displacement Coordinates Gradient of Static Image The differential force between moving and static images In the above equation the gradient information that drives the deformation is inefficiently taken from the static image only. Diffusion is bi-directional, ‘demons’ at any point in the image space forces allows a deformable ‘moving’ object to diffuse into a static object, forces ‘static’ object to diffuse into the ‘moving’ object. Passive force + Active force He Wang, Lei Dong, Jennifer O'Daniel, Radhe Mohan, Adam S Garden, K Kian Ang, Deborah A Kuban, Mark Bonnen, Joe Y Chang and Rex Cheung Validation of an accelerated 'demons' algorithm for deformable image registration in 8 radiation therapy Phys. Med. Biol. 50 2887 -2905 , 2005

Lukas-Kanade Optical Flow Method – small motion: – suppose we take the Taylor series expansion of I: 9

Lukas-Kanade Optical Flow Method Lukas & Kanade (1981) • How to get more equations for a pixel? – Basic idea: impose additional constraints • most common is to assume that the flow field is smooth locally • one method: assume the pixel’s neighbors have the same (u, v) – If we use a 5 x 5 window, this gives us 25 equations per pixel! 10

Synthetic Atmospheric Turbulence-like Effect r φ R θ 11

Deformation correction Experiments Reference Synthetically degraded Corrected with Demons 12

Deformation correction Experiments Reference Synthetically degraded Corrected with LK 13

Future work: A new method for non-rigid registration Using a force field that attracts points close in position and attribute (intensity) 14

The Gaussian Criterion Datasets: Gaussian Field: Criterion: 15