Shadow removal algorithms Shadow removal seminar Pavel Knur

Shadow removal algorithms Shadow removal seminar Pavel Knur

Deriving intrinsic images from image sequences Yair Weiss July 2001.

History • “intrinsic images” by Barrow and Tenenbaum , 1978

Constraints • Fixed viewpoint • Works only for static objects • Cast shadows

Classic ill-posed problem • Denote – – – the input image the reflectance image the illumination image Number of Unknowns is twice the number of equations.

The problem Given a sequence of T images in which reflectance is constant over the time and only the illumination changes, can we solve for a single reflectance image and T illumination images ? Still completely ill-posed : at every pixel there are T equations and T+1 unknowns.

Maximum-likelihood estimation • Log domain :

Assumptions When derivative filters are applied to natural images, the filter outputs tend to be sparse.

Laplacian distribution Can be well fit by laplacian distribution

Claim 1 Denote : • N filters – • Filter outputs – • Filtered reflectance image – ML estimation of filtered reflectance image is given by

Estimated reflectance function Recover ML estimation of r is reversed filter of

ML estimation algorithm

ML estimation algorithm – cont. • Ones we have estimated

Claim 2 • What if does not have exactly a Laplasian distribution ? Let Then estimated filtered reflectance are within with probability at least:

Claim 2 - proof If more than 50% of the samples of are within of some value, then by definition of median, the median must be within of that value.

Example 1 • Einstein image is translated diagonally • 4 pixels per frame

Example 2 • 64 images with variable lighting from Yale Face Database

Illumination Normalization with Time. Dependent Intrinsic Images for Video Surveillance Y. Matsushita, K. Nishito, K. Ikeuchi Oct. 2004

Illumination Normalization algorithm • Preprocessing stage for robust video surveillance. • Causes – Illumination conditions – Weather conditions – Large buildings and trees • Goal – To “normalize” the input image sequence in terms of incident lighting.

Constraints • Fixed viewpoint • Works only for static objects • Cast shadows

Background images Input images • Remove moving objects from the input image sequence Off-line Background images

Estimation of Intrinsic Images Input images Denote • • • Off-line input image time-varying reflectance image time-varying illumination image reflectance image estimated by ML illumination image estimated by ML • Filters • Log domain Background images Estimation of Intrinsic Images

Estimation of Intrinsic Images – cont. Input images Off-line Background images Estimation of Intrinsic Images • In Weiss’s original work • The goal is to find estimation of and

Estimation of Intrinsic Images – cont. Input images Basic idea: • Estimate time-varying reflectance components by canceling the scene texture from initial illumination images Define: Off-line Background images Estimation of Intrinsic Images

Estimation of Intrinsic Images – cont. Input images Finally : Off-line Background images Estimation of Intrinsic Images Where : is reversed filter of

Shadow Removal Input images Denote Off-line - background image - illuminance-invariant image Background images Estimation of Intrinsic Images

Illumination Eigenspace Input images • PCA – Principle component analysis Basic components Illumination Eigenspace Off-line Background images Estimation of Intrinsic Images

Illumination Eigenspace – cont. Input images • Average is Off-line Background images • P is Mx. N matrix where Illumination Eigenspace – N – number of pixels in illumination image – M – number of illumination images • Covariance matrix Q of P is Estimation of Intrinsic Images

Direct Estimation of Illumination Images Input images • Pseudoillumination image Off-line Background images • Direct Estimation is Illumination Eigenspace • Where – F is a projection function onto the j’s eigenvector - Estimation of Intrinsic Images

Direct Estimation of Illumination Images Input images • Results Off-line Background images Illumination Eigenspace Estimation of Intrinsic Images

Shadow interpolation Input images probability density function cumulative probability function shadowed area lit area Illumination Eigenspace Shadow Interpolation mean optimum threshold value Off-line Background images Estimation of Intrinsic Images

The whole algorithm Input images Off-line Background images Illumination Eigenspace Estimation of Intrinsic Images Shadow Interpolation Illumination Images / Normalization

Example

Questions ?

References [1] Y. Weiss, ”Deriving Intrinsic Images from Image Sequences”, Proc. Ninth IEEE Int’l Conf. Computer Vision, pp. 68 -75, July 2001. [2] Y. Matsushita, K. Nishito, K. Ikeuchi, “Illum ination Normalization with Time. Dependent Intrinsic Images for Video Surveillance”, Oct. 2004.