Accurate Image Based Relighting through Optimization Pieter Peers

Accurate Image Based Relighting through Optimization Pieter Peers Philip Dutré Department of Computer Science K. U. Leuven, Belgium

What is Image Based Relighting? Basis Images Relit Image Processing & Relighting

Previous Work • The Light Stage (Debevec et al. ’ 00) Images: ACM SIGGRAPH ‘ 00

Previous Work • Environment Matting (Zonker et al. ’ 99, Chuang et al. ’ 00) Images: ACM SIGGRAPH ’ 99

Previous work Diffuse Materials Specular Materials + - Sufficient angular sampling Sparse sampling of light source directions - + The Light Stage Environment Matting Large span of incoming light directions • Matusik et al. EGWR’ 02 Bundled incoming light directions

Three-step algorithm 2 1 Basis Image Recording 3 Relit image = Captured reflectance + Light-map Reflectance field calculation

Basis Image Recording

Basis Image Recording

Basis Image Recording . . . HDR photographs

Practical Setup • HDR photographs • Monitor calibration • 4 x 4 grid, 9 patterns per lightpatch • ± 600 basis images

Three-step algorithm 2 1 Basis Image Recording 3 Relit image = Captured reflectance + Light-map Reflectance field calculation

Support and reflection coefficient

Support and reflectance Pixel … Patterns … Reflection Coefficient Resulting Value = → Error 1 = → Error 2 × Rpix, patch Find … = → Errorn-1 = → Errorn and Rpix, patch that minimize: Errork

Support and reflectance: examples cli ck left top back right

Three-step algorithm 2 1 Basis Image Recording 3 Relit image = Captured reflectance + Light-map Reflectance field calculation

Re-lighting For each pixel: Pixel value = ∑ support i Filteri(Light. Map) × Rpix, i

Results A B C D

Results A B C D

Results A B C D

Results A B C D

Comparison Stochastic ray tracing Relit from rendered basis images

Comparison Stochastic ray tracing Relit from rendered basis images

Comparison Stochastic ray tracing Relit from rendered basis images

Conclusion • Accurate relighting: – Diffuse materials – Specular materials • Combine strengths from: – The Light Stage – Environment matting

Future Work • Alternative emitting devices • Investigate other patterns & optimization algorithms

Acknowledgements • Frank Suykens and Vincent Masselus • Some environment maps from: http: //www. debevec. org/Probes

Thank you… Questions?