Independent Component Analysis ICA YaLing Yao Motivation A

Independent Component Analysis (ICA) Ya-Ling Yao

Motivation • A Method: find underlying factors or components form multi-dimensional statistical data • Distinguishes : looks for components that are both statistically independent and nongaussian data

1. ORIGINAL SOUND SOURCES 2. SAMPLES AT THE COCKTAIL PARTY 3. FOUND SOURCES Apply ICA to separate the samples sound sources http: //research. ics. aalto. fi/ica/cocktail_en. cgi

PCA and ICA • Multi-dimensional statistical – PCA and ICA: reduce dimensions • Difference: – PCA: with a Gaussian model – ICA: with non-Gaussian model – PCA: Vector are orthogonal – ICA: Vector are not orthogonal

PCA and ICA http: //gael-varoquaux. info/science/ica_vs_pca. html

ICA mathematical approach • xi = ai 1 s 1 + ai 2 s 2 + … + ainsn, for all i = 1, …, n Giving: observation “x” Find: Original independent components “s” and Associated linear combination “aij”

• Also written as • Independent components are latent variables meaning they cannot be directly observed • The mixing matrix is assumed to be unknown. • All observe is the random vector x, and must estimate both A and s by using it. • This must be done under as general assumption as possible.

Assumptions of ICA •

Limitation of ICA •

Central Limit theorem • Mix many non-gaussian independent components then source distribution will tend to become Gaussian distribution. – Matlab: we have 10000 data length and generate random data

1 random variable

2 random variables

50 random variables

Measure non-Gaussian •

Measure non-Gaussian • Kurtosis=0 Gaussian • Kurtosis>0 Super. Gaussian – nongaussian • Kurtosis<0 Sub. Gaussian – nongaussian

ICA processes • 1. centering data (remove mean) • 2. whitening process (sphere data) – 2. 1 Uncorrelate variables – 2. 2 Scale variables so that their variance =1 – Whitening + non-gaussianity independent

MLE - ICA Algorithm •

MLE - ICA Algorithm (con’t) •

Application • Data: MEG data (Magnetoencephalography) • Data description: – Test: • Eye artifacts: – ask person to “blink” and to make “horizontal saccades” • Muscle artifacts: – Asked to bite teeth for as long as 20 seconds. • Other artifact: – Cardiac cycle – Subset: 12 subset of MEG signals xi(t)

Figure: Samples of MEG signals, showing artifacts produced by blinking, saccades, biting and cardiac cycle. For each of the 6 positions shown, the two orthogonal directions of the sensors are plotted.

Result There are 9 ICA found from the recorded data but I am only looking at ICA that could identify the Eye artifacts, Muscle artifacts and Cardiac cycle. Clearly due to the muscular activity originated fro the biting

Result Showing Horizontal eye movement IC 3 and the eye blinks IC 5

Result IC 4 is clearly extracted to represents the cardiac artifact. Conclusion: This result are showing clearly that using ICA technique, and Fast. ICA algorithm, it is possible to isolate both eye movement and eye blinking artifacts and other artifacts from signals

Other Applications • Finding Hidden Factors in Financial Data • Currency exchange rate or daily returns of stocks may have some common underlying factors • Reducing Noise in Natural Image • Based on the ICA decomposition, removing noise from images corrupted with additive Gaussian noise to find ICA filters for natural images • Telecommunications (new) • Useful to separate the user’s own signal from the interfering other users’s signal in CDMA (Code-Division Multiple Access)

References • Aapo Hyvärinen and Erkki Oja Neural Networks Research Centre Helsinki University of Technology P. O. Box 5400, FIN-02015 HUT, Finland Neural Networks, 13(4 -5): 411 -430, 2000. Retrieved from http: //www. cs. helsinki. fi/u/ahyvarin/papers/NN 00 new. pdf • Aapo Hyvarinen, Juha Karhunen, and Erkki Oja (2001). Independent Component Analysis. Retrieved from http: //www. cs. helsinki. fi/u/ahyvarin/papers/bookfi nal_ICA. pdf