Photorealistic Rendering of Rain Streaks Kshitiz Garg Shree

Photorealistic Rendering of Rain Streaks Kshitiz Garg Shree K. Nayar Department of Computer Science Columbia University SIGGRAPH Conference July 2006, Boston, USA Sponsors: NSF, ONR

A Rain Scene from “The Matrix Revolutions”

A Rain Scene from “The Matrix Revolutions” Frame 1 Frame 2 Frame 3

Rendering Streaks using Simple Brightness Model Commercial Software Heuristic-based [Starik and Werman 03] Hybrid-Spectral Method[Langer et al. 04] ATI Toy. Shop Demo [N. Tataturschuk et al. 05]

Rendering Streaks using Hand Drawn Textures Frame from “The Matrix Revolutions” Original Image Courtesy Warner Bros. Hand Drawn Textures Original Image Courtesy Andy Lomas

Study of Visual Appearance of Rain Streaks • Rain Streak Appearance Model • Rain Streak Texture Database • Image-Based Rain Rendering Algorithm

Oscillations in a Rain Drop [Beard and Kubesh 1989 Andsager et al. 1992] Turbulent Flow of air A Falling Rain Drop

Oscillations in a Rain Drop [Beard and Kubesh 1989 Andsager et al. 1992] Oblate-Prolate Mode Transverse Mode Oscillations in a Rain Drop

Appearance of a Rain Drop Specular Reflection N Refraction Internal Reflection

Appearance of a Non-Oscillating Rain Drop N Non-Oscillating Rain Drop Instant Time Shots Falling Drop Rain Streak

Appearance of an Oscillating Rain Dro N

Rain Streak Appearance Model [Beard and Kubesh 1989 Andsager et al. 1992] Shape of Rain Drop: ? Fall Direction fdrop= (00 – 3600) Not Rotationally Symmetric ? Oblate-Prolate Mode Transverse Mode

Measurement of Rain Streak Appearances Experimental Setup Drop generator Height: 15 m y, ydrop Lighting and Viewing Configuration qlight = (500, 900, 1300) flight = (100 , 300, …, 1700) qview = (700, 900, 1100) Oscillation Parameters qvie w x fdrop xdrop flig ht qlig ht -z (A 2, 0 A 3, 1 fdrop) =(10 images per light & view)

Estimating the Oscillation Amplitudes qview qligh tf lig 110 130 90 90 10 70 50 30 10 150 130 30 Measured ht Estimation Result Oscillation Amplitudes: 10 70 50 90 110 50 170 130 30 170 50 90 90 110 50 130 30

Comparison of Actual vs. Rendered Rain Streaks qview qligh tf lig Rendered Measured ht 110 130 90 90 10 70 50 30 10 150 130 30 10 70 50 90 110 50 170 130 30 170 50 90 90 110 50 130 30

Precomputed Database of Streak Textures Factors Affecting the Appearance Render Offline (Database) Lighting and Viewing Direction Oscillations Parameters Rain Drop Size Render Online (Algorithm) Source and Camera Distance Camera Parameter Database contains 6300 HDR Rain Streak Textures (400 MB) ( Publicly available: Please send email to rainstreaks@cs. columbia. edu )

Rain Rendering Pipeline User Input Rain Dynamics Streak texture (Source 1) Camera and Rain Parameters Light Source Properties Coarse Depth Map Render Novel Streak Textures Perspective Effects Scale Rotate Raindrop Distribution and Trajectories Using Particle System + Streak texture (Source n) + Streak texture (Ambient) Scene Image/Video Render Streak Appearance Rain Streak Database Camera Effects Exposure Crop/ Blend Defocus Blur Scene Image/Video With Rain

Rendered Rain with Varying Light Direction -150 o Light Source Direction

Constant Brightness Streak Model

Comparison with Constant Brightness Model Streak Appearance Model Constant Brightness Model -150 o Light Source Direction

Night Scene with Isotropic Source Coarse Depth Map Image of the Scene

Night Scene with Isotropic Source Rendered Rain

Night Scene with Isotropic Source Defocus Effects

Scene with Colored and Anisotropic Sources Coarse Depth Map Scene with Multiple Sources

Scene with Colored and Anisotropic Sources Rendered Rain

Scene with Colored and Anisotropic Sources Streak Appearance Model Constant Brightness Model

Scene with Changing Sky Illumination Input Scene Video

Scene with Changing Sky Illumination Rendered Rain

A Scene with Moving Light Sources Input Scene Video

A Scene with Moving Light Sources Rendered Video

Future Extensions • Real-time Implementation of Rendering Algorithm • Incorporating the Effects of Wind and Turbulence

Summary Rain Streak Appearance Model Rain Streak Texture Database ( Publicly available: rainstreaks@cs. columbia. edu ) Image-Based Rain Rendering Algorithm