Analysis of Spatiochromatic Decorrelation for Colour Image Reconstruction

Analysis of Spatio-chromatic Decorrelation for Colour Image Reconstruction Mark S. Drew and Steven Bergner School of Computing Science, Simon Fraser University, Canada {mark/sbergner}@cs. sfu. ca Color Imaging 2004

I. Overview - Use of PCA vs. ICA — what’s the difference? - How do you do ICA? - What does this have to do with images? - The objective: best characterize image blocks using ICA on color image block data == spatio (blocks are 16 x 16, say)chromatic (x 3); assign bits in bit allocation according to the importance of each ICA coefficient data compression. Color Imaging 2004 2 2

Best characterize image colour and spatial information. Colour: we think of using PCA (Principal Component Anaysis): discover main colour axes. Is this best, given our objective? Spatial: use spatial Fourier filters? Gabor wavelets? Etc. Here, we’ll use ICA (Independent Component Anaysis) to derive best colour and spatial decomposition at once, for decorrelation, compression, and reconstruction. Color Imaging 2004 3 3

II. ICA �What is it? ICA is a form of “Blind Source Separation” To explain, consider audio signals (in an Imaging conference!). Consider 2 speakers, and 2 microphones: s 1 s 2 x 1 -sources -data Color Imaging 2004 4 4

Can we disentangle s 1, s 2 from measured data x 1, x 2 ? == The “cocktail party problem”. An example: Color Imaging 2004 5 5

ICA: Order and sign not determined. Color Imaging 2004 6 6

What about PCA? Writing the signals in terms of reduced set of sources s 1, s 2, s 3, . . . , for higher-dimensional data, we can do a better job in compression. Color Imaging 2004 7 7

III. ICA �How to do it? Model: (x was 2 x. N in the audio example. ) mixing matrix separating matrix Color Imaging 2004 8 8

Driving idea for finding sources: s 1, s 2 are statistically independent == information about one gives no knowledge re. the other. Not just uncorrelated: covariance = 0 ==PCA Color Imaging 2004 9 9

If independent as well, the pdf is separable: marginal pdf’s joint pdf which implies for any functions , ! useful for solving. Color Imaging 2004 10 10

So, to do ICA, start with uncorrelated signals (using PCA) == simplifies. Main tool: Non-Gaussian is independent. Central Limit Theorem: the sum of two independents is more like a Gaussian than is either one. So we have sums . To get s, make a linear combination of x’s that is as non-Gaussian as possible. Color Imaging 2004 11 11

One way: (…many others) A Gaussian has zero kurtosis. For zero mean y, Rescale y to variance=1: just use We seek a signal that maximizes kurtosis. Color Imaging 2004 12 12

Algorithm “whiten” the data: zero mean, + linear transform to make uncorrelated, variance=1. First, PCA: orthogonal U with In the new coordinate system, Why? Now with orthogonal simpler to search for. Color Imaging 2004 13 13

Algorithm -whiten x -we seek a column w of orthogonal W, with that maximizes kurtosis: , Euler eqn. : Code 1. Initialize w randomly, with 2. 3. 4. stop when Color Imaging 2004 14 14

Matlab Color Imaging 2004 15 15

IV. ICA for Images Previous work: Greyscale and colour imagery using PCA and ICA. For colour images, x could be 3 -vector pixels. But get spatial as well if use n n tiles (nice illustration in Süsstrunk et al. , CGIV’ 04 [using PCA on raw CFA data]) We show here that compression is better using ICA+colour+spatial info. Color Imaging 2004 16 16

ICA (162 x 1 greyscale data) 16 x 16 greyscale tiles ICA finds “sparse” features: localization in sp Color Imaging 2004 17 17

With colour: (no spatial information) PCA vs. ICA (3 x 1 data) Color Imaging 2004 18 18

PCA vs. DCT (4 x 4 -less axis-aligned -ordering by variance -accounted-for is different: pure colour axes appear first PCA x 3 data) (4 x 4 x 3) -Colour: luminance, blue-yellow, red-green -pure colour axes appear later, after luminance frequencies -separates colour from luminance DCT (4 x 4 x 3) Color Imaging 2004 19 19

PCA vs. ICA -localization in frequency PCA (4 x 4 x 3) again -colour less separate from spatial information -combined localization in space and frequency -patterns not rectangular more like Gabor functions (Gaussian -modulated sine functions) ICA Color Imaging 2004 (4 x 4 x 3) 20 20

ICA (4 x 4) ICA (8 x 8) ICA (16 x 16) Color Imaging 2004 (5 x 5) 21 21

Colour vs. Greyscale: Compression performance Colour Be tt er qu al it y (Generic basis) Greyscale SNR - Higher reconstruction quality (SNR) for larger patches - Colour has better quality than grey, at equal compression Color Imaging 2004 22 22

ICA vs. PCA (Specific basis: image = ) PCA ICA Be tt er qu al it y - ICA much better than PCA: higher compression for same SNR - ICA increased quality with larger patches, for equal compression Color Imaging 2004 23 23

ICA vs. PCA A. ICA does better separating axes such that they influence each other least better entropy coding B. Colour aids in compression C. Large patch sizes and low rate encoding At equal compression, SNR (quality) better for ICA Color Imaging 2004 24 24

ICA vs. PCA: Image reconstruction (compression ratio: 1: 12) ICA PSNR= 35. 55 DCT: PSNR= 31. 97 Color Imaging 2004 25 25

Another image 7: 1 ICA PSNR= 39. 69 DCT: PSNR= 31. 40 Color Imaging 2004 Orig ICA DCT --blocking 26 26

The Future: Video Bases ICA (6 x 6 x 6) PCA (6 x 6 x 6) [submitted] Color Imaging 2004 27 27