Vision in Bad Weather Shree Nayar and Srinivasa

Vision in Bad Weather Shree Nayar and Srinivasa Narasimhan Computer Science Columbia University ICCV Conference Korfu, Greece, September 1999 Sponsors: NSF

Vision and Bad Weather Clear Day Dense Rain Fog Noon Haze

Weather Conditions and Particle Sizes ( Mc. Cartney, 1975 ) CONDITION PARTICLE TYPE AIR Molecule HAZE Aerosol FOG Water Droplet CLOUD Water Droplet RAIN Water Drop RADIUS ( m) CONCENTRATION (cm -3)

Particle Scattering Mechanisms ( Mie 1908 ) • Single Scattering : Incident Beam Size: 0. 01 Size: 0. 1 Size: 1 • Multiple Scattering : First Order Third Order Incident Beam Second Order

Atmospheric Optics • Overviews : Middleton 1952 , Mc. Cartney 1976 • Haze : Hulburt 1946 , Hidy 1972 • Fog : Koshmeider 1924 , George 1951 , Myers 1968 • Clouds and Rain : Laws 1943 , Houghton 1951 , Mason 1975 • Snow : Ohtake 1970 • Vision : Koenderink & Richards 1992 , Cozman & Krotkov 1997

Attenuation Model ( Mc. Cartney, 1975 ) Scattering Medium Attenuated Exiting Beam Collimated Incident Beam X=d Unit Cross Section dx X=0 Total Flux Scattered by Lamina (dx) : Total Scattering Coefficient

Direct Transmission Beam Irradiance at Distance d : (Bouguer’s Law, 1729) Attenuation of Diverging Beams : ( Allard’s Law, 1876 ) Optical Thickness :

Airlight Model Sunlight Diffuse Skylight ( Koschmieder, 1924 ) Object d. V Observer dw dx x d Diffuse Ground Light Radiant Intensity of Volume (d. V) :

Airlight Image Irradiance due to Volume d. V : Image Irradiance due a Path of Length d : Horizon Radiance

Light Sources in Night Fog No Environmental Illumination Street Lamp Window Camera Attenuation Model : Image Irradiance for Camera Response s( ) :

Depth of Sources from Two Night Images Two Weather Conditions : , ( Unknown ) Ratio of Image Irradiances : Difference in Optical Thickness (DOT) : Relative Depths from Two Sources (i, j):

Depth of Sources from Attenuation Computed Source Locations Clear Day Clear Night Foggy Night

Structure from Airlight Using a Single Image Weather Condition : ( Dense Haze or Fog) Airlight Model for Camera Response s( ) : Image Irradiance Relative to the Horizon : Optical Thickness :

Structure from Airlight Mountain Range Urban Scene Foggy Day Image Computed Depth Map 3 D Structure Computed Depth Map Stereo ? Motion ? 3 D Structure

Illumination Occlusion Problem Scene Point Observer

Dichromatic Atmospheric Scattering Model Image Irradiance and Wavelength : Direct Transmission Airlight Surface Radiance Horizon Radiance Rayleigh’s Law : For Fog : ( Middleton 1952 )

Dichromatic Atmospheric Scattering Model Finite-Dimensional Color Space : B E dt E E a (volume) G R Final Model : (surface + volume)

True Color from Two Images Clear Day Image Defogged and Enhanced Windows Foggy Day Image

Chromatic Decomposition Using Two Images Recovered True Color Clear Day Image Computed Depth Map Foggy Day Image 3 D Structure

Summary • No Escape from Bad Weather • Image Processing will Not Suffice • Bad Weather can be Good for Vision