Multispectral Image Invariant to Illumination Colour Strength and

Multispectral Image Invariant to Illumination Colour, Strength, and Shading Mark S. Drew and Amin Yazdani Salekdeh School of Computing Science, Simon Fraser University, Vancouver, BC, Canada {mark/ayazdani}@cs. sfu. ca

Table of Contents Introduction RGB Illumination Invariant Multispectral Image Formation Synthetic Multispectral Images Measured Multispectral Images Conclusion 2

Introduction Invariant Images – RGB: Information from one pixel, with calibration Information from all pixels – use entropy New Multispectral data: Information from one pixel without calibration, but knowledge of narrowband sensors peak wavelengths 3

RGB Illumination Invariant Removing Shadows from Images, ECCV 2002 Graham Finlayson, Steven Hordley, and Mark Drew 4

RGB… An example, with delta function sensitivities B Narrow-band (delta-function sensitivities) P R W G Y Log-opponent chromaticities for 6 surfaces under 9 lights

RGB… Deriving the Illuminant Invariant Log-opponent chromaticities for 6 surfaces under 9 lights Rotate chromaticities This axis is invariant to illuminant colour

RGB… An example with real camera data Normalized sensitivities of a SONY DXC-930 video camera Log-opponent chromaticities for 6 surfaces under 9 different lights

RGB… Deriving the invariant Log-opponent chromaticities Rotate chromaticities The invariant axis is now only approximately illuminant invariant (but hopefully good enough)

Multispectral Image Formation Illumination : motivate using theoretical assumptions, then test in practice Planck’s Law in Wien’s approximation: Lambertian surface S( ), shading is , intensity 9 is I Narrowband sensors qk( ), k=1. . 31, qk( )= ( k) Specular: colour is same as colour of light (dielectric):

Multispectral Image Formation … To equalize confidence in 31 channels, use a geometric-mean chromaticity: Geometric Mean Chromaticity: with 10

Multispectral Image Formation … surface-dependent sensor-dependent illuminationdependent So take a log to linearize in (1/T) ! 11

Multispectral Image Formation … Logarithm: known because, in special case of multispectral, *know* k ! Only sensor-unknown ! ( spectral-channel gains) 12 is

Multispectral Image Formation … If we could identify at least one specularity, we could recover log k ? ? Nope, no pixel is free enough of surface colour . So (without a calibration) we won’t get log k, but instead it will be the origin in the invariant space. Note: Effect of light intensity and shading removed: 31 D 30 -D Now let’s remove lighting colour too: we know 31 -vector (ek – e. M) (-c 2/ k - c 2/ M) 13 Projection to (ek – e. M) removes

Algorithm: -Form 31 -D chromaticity k - Take log - Project to (ek – e. M) using projector Pe

Algorithm: What’s different from RGB? For RGB have to get “lighting-change direction” (ek – e. M) either from (i)calibration, or (ii) internal evidence (entropy) in the image. (iii)For multispectral, we know (ek – e. M) !

First, consider synthetic images, for understanding: Surfaces: 3 spheres, reflectances from Macbeth Color. Checker Camera: Kodak DSC 420 31 sensor gains qk( ) Carry out all in 31 -D, but show as camera would see it. 16

Synthetic Images Under blue light, P 10500 17 Under red light, P 2800 shading, for light 1, for light 2

Synthetic Images Original: not invariant Spectral invariant 18

Measured Multispectral Images Under D 75 Invt. #1 19 Under D 48 Invt. #2

Measured Multispectral Images In-shadow, In-light After invt. processing 20

Measured Multispectral Images 21

Measured Multispectral Images 22

Measured Multispectral Images 23

Conclusion A novel method for producing illumination invariant, multispectral image Successful in removing effects of Illuminant strength, colour, and shading • Next: removing shadows from 24 remote-sensing data.

Thanks! Funding: Natural Sciences and Engineering Research Council of Canada 25 Multispectral Images Invariant to Illumination Colour, Strength and Shading