Flutter Shutter Video Camera for Compressive Sensing Jason

Flutter Shutter Video Camera for Compressive Sensing Jason Holloway Aswin Sankaranarayanan Ashok Veeraraghavan Salil Tambe April 28, 2012

Image formation model Low-speed images work for static scenes

Video capture of high-speed scenes Scaled image on the sensor 33 ms open shut

Video capture of high-speed scenes Blurring in dynamic areas High spatial resolution in static areas

Rich video on a voxel budget 1 Megapixel x 30 fps 175 x 1000 fps � 30 million voxel budget �Increasing fps decreases light throughput

Video formation model Optical coding

Optical coding approaches Spatial-muliplexing Per pixel shutter control Single Pixel Camera [Wakin et al. 2006] P 2 C 2 [Reddy et al. 2011] -LCOS -Per pixel coded y exposure Per pixel sensor control x Global shutter control FSVC – Global shutter control Flexible Voxels Gupta et al. 2011] CPEV [Hitomi et al. 2010, t -Implement on CMOS -Single bump per pixel x y per frame t t y x Exposure

Scene assumptions Periodicity Linear dynamical systems Coded Strobing [Veeraraghavan et al. 2011] CD-LDS [Sankaranarayanan et al. 2010] Video Removed 80 x Linear with known velocity Flutter Shutter [Raskar et al. 2006] 20 x-50 x General motion Flexible Voxels [Gupta et al. 2010] P 2 C 2 [Reddy et al. 2011] CPEV [Hitomi et al. 2011] FSVC 6 x-16 x

Recovering high-speed video �Locally linear motion model Union of subspaces (Uo. S) �General motion TV minimization

Union of subspaces (Uo. S) t t y 521 velocities, 40 x y angles and 13 speeds High-speed PCA subspace approx. A 18 x 24 patch can be expressed using 324 dimensional subspace

Union of subspaces (Uo. S) High-speed subspace Low-speed subspace Patch based reconstruction

Patch recovery with Uo. S prior �Speed = 1. 38 pixels/high-speed frame �Direction = 61° Speed (pixels/frame)

Recovery with Uo. S prior PSNR: 35. 5 d. B Video Removed

TV minimization recovery �

Union of subspaces recovery (6 x) �Hairnets advertisement placard moving right PSNR: 40. 6 d. B Video Removed

TV minimization recovery (6 x) �Dancer clapping causes a chalk cloud to form PSNR: 28. 7 d. B Video Removed

Experimental setup �Point. Grey Machine Vision cameras were used to simulate FSVC �Flea 3 grayscale camera operating at 8 fps

Real data results 6 x (Uo. S) Observed frames Recovered video (6 x) Video Removed

Conclusions Reconstruction quality Global shutter control suffices for high speed video recovery FSVC CPEV P 2 C 2 Hardware complexity

Thank you �Questions?