The Role of Bright Pixels in Illumination Estimation

The Role of Bright Pixels in Illumination Estimation Hamid Reza Vaezi Joze Mark S. Drew Graham D. Finlayson Petra Aurora Troncoso Rey School of Computer Science Simon Fraser University School of Computer Sciences The University of East Anglia November 2012

Outline Motivation Related research Extending the white-patch hypothesis The effect if bright pixels in well-known methods The bright-pixels framework Further experiment Conclusion 2

Motivation White-Patch method One of the first colour constancy methods Estimates the illuminant colour by the max response of three channels Few researchers or commercial cameras use it now Recent research reconsider white patch Local mean calculation as a preprocessing can significantly improve [Choudhury & Medioni (CRICV 09)] [Funt & Li (CIC 2010)] Analytically, the geometric mean of bright (specular) pixels is the optimal estimate for the illuminant, based on dichromatic model 3 [Drew et al. (CPCV 12)]

Bright Pixels Light Source Just a bright surface White surface Highlights 4

Previous Research White Patch Local mean calculation as a preprocessing step for White Patch Using Specular Reflection Specular reflection colour is same as the illumination within a Neutral Interface Reflection It usually includes the bright areas of image Illumination estimation method Intersection of dichromatic planes [Tominaga and Wandell (JOSA 89)] Intersection of the lines generates by chromaticity values of pixels of each surface in the CIE chromaticity diagram by [Lee (JOSA 86)] 5 Extending Lee’s algorithms by constraint on the colours of illumination

Grey-based illumination estimation Grey-world The average reflectance in the scene is achromatic Shade-of-grey Minkowski p-norm Grey-edge The average of the reflectance differences in a scene is achromatic 6

Extending the White Patch Hypothesis Let us extend white-patch hypothesis that there is 7 always include any of: white patch, specularities, or light source in an image Gamut of bright pixels, in contradistinction to maximum channel response of the White-Patch method, which include the brightest pixels in the image Removing clipped pixels (exceed 90% of the dynamic range) Define bright pixels as the top T % of luminance given by R+G+B. What is the probability of having an image without strong highlights, source of light, or white surface in the real world?

Simple Experiment whether or not the actual illuminant colour falls inside the 2 D gamut of top 5% brightness pixels SFU Laboratory Dataset : 88. 16% Color. Checker : 74. 47% Specularity Grey. Ball : 66. 02% FAIL White surface 8

The Effect of Bright Pixels on Grey-base methods Experiment the effect of bright pixels Run grey-based method for the top 20% brightness pixels in each image, and compare to using all image pixels (colour) Color. Checker Dataset 9 Using one fifth of the pixels performance is better or equal

The Effect of Bright Pixels on Gamut Mapping method White-patch gamut and canonical white-patch gamut introduced [Vaezi Joze & Drew (ICIP 12)] White-patch gamut is the gamut of top 5% bright pixels in an image • Adding new constraints based on the white-patch gamut to standard Gamut Mapping constraints outperforms the Gamut Mapping method and its extensions. 10 Canonical gamut vs. WP canonical gamut

The Bright-Pixels Framework If these bright pixels represent highlights, a white surface, or a light source, they approximate the colour of the illuminant Try Mean, Median, Geomean, p -norm (p=2, p=4) for top T% brightness 11

The Bright-Pixels Framework A local mean calculation can help: Resizing to 64 × 64 pixels by bicubic interpolation Median filtering Gaussian blurring filter Color. Checker Dataset It does not help so much on these images 12

Dataset 1. SFU Laboratory [Barnard & Funt (CRA 02)] 321 images under 11 different measured illuminants 1. Reprocessed version of Color. Checker [Gehler et al. (CVPR 08)] 568 images, both indoor and outdoor 2. Grey. Ball [Cieurea & Funt (CIC 03)] 11346 images extracted from video recorded under a wide variety of imaging conditions 3. HDR dataset [Funt et al. (2010)] 105 HDR images 13

The Bright-Pixels Method 1. Remove clipped pixels 2. Do local mean {no, Median, Gaussian, Bicubic } 3. Select top T% brightness pixels Threshold = {. 5%, 1%, 2%, 5%, 10%} 4. Estimate illuminant by shade of grey eq. p = {1, 2, 4, 8} 5. if the estimated illuminant is not in the possible illuminant gamut use grey-edge 14

Further Experiment Comparison with well-known colour constancy methods 15

Optimal parameters p T blurring SFU Laboratory Dataset 2 . 5 % no Color Checker Dataset 2 2% Gaussian Grey. Ball Dataset 2 1% no HDR Dataset 8 1% Gaussian for high resolution images and no blurring for lower resolution images Even. 5% threshold is enough for in-laboratory images, for real images threshold should be 12% 16

Conclusion Based on current datasets in the field we saw that the simple idea of using the p-norm of bright pixels, after a local mean preprocessing step, can perform surprisingly competitively to complex methods. Either the probability of having an image without strong highlights, source of light, or white surface in the real world is not overwhelmingly great or the current color constancy datasets are conceivably not good indicators of performance with regard to possible real world images. 17

Questions? Thank you. 18