Learning StructureAndMotionAware Rolling Shutter Correction Bingbing Zhuang QuocHuy
Learning Structure-And-Motion-Aware Rolling Shutter Correction Bingbing Zhuang, Quoc-Huy Tran, Pan Ji, Loong Fah Cheong, Manmohan Chandraker
Rolling shutter cameras are everywhere …
Rolling Shutter Effect Sequential Exposure Mode 3 D Point Camera motion between scanlines
Rolling Shutter Distortion Our Correction
Difficulties • Complexity of RS geometry Two-view approach [Dai et. al. CVPR 17], [Zhuang et. al. ICCV 17] • Degeneracy of RS geometry q RS two-view Sf. M under pure translation: a critical configuration (contribution 1) q Our single-view approach (contribution 2) ü Data-driven with CNN ü Structure-And-Motion-Aware
Critical Configuration Analysis Proposition. RS two-view Sf. M under pure translational camera motion is degenerate. Ø Infinitely many solutions exist for per-scanline camera positions and depth. See paper for a proof Radiating Pattern
Our Single-view Approach Network Architecture Rectification RS GS Camera motion between scanlines
Training Data Generation Synthesize: GS RS 3 D points RS image GS depth Camera motion between scanlines RS depth
An Ambiguity In Training Data Preprocessing Different sizes Same size Resize?
An Ambiguity In Training Data Preprocessing VS Vertical Resizing Similar to the Bas-Relief ambiguity in classical Sf. M
Result: Synthetic RS Image RS Ground Truth GS Ground Truth Warping Flow
Result: Synthetic RS Image GT Ours 2 DCNN MH (1) MH, [Purkait et. al. ICCV 17] (2) 2 DCNN, [Rengarajan et. al. CVPR 17] (3) Ours
Result: Real RS image Input MH 2 DCNN Ours
Conclusion • RS geometry may suffer from degeneracy. • Geometrical guidance is the key to improving RS correction. Rectification • Future work: Motion deblurring For more details, visit www. neclabs. com/research-departments/media-analytics/.
- Slides: 14