Blind Source Separation Independent Component Analysis By Advisor

Blind Source Separation & Independent Component Analysis By: Advisor: Soroosh Mariooryad Dr. Sameti BSS & ICA Speech Recognition - Spring 2008 1

History Observation in 1982: The angular position and the angular velocity of a joint is represented by two nervous signals f 1(t) and f 2(t), each one is a linear combination of position and velocity: At each instant the nervous system knows p(t) and v(t) must be recoverable form f 1(t) and f 2(t) BSS & ICA Speech Recognition - Spring 2008 2

Blind Source Separation Si: Original source(assumed to be Independent) Xi: Received (mixed) signals. Yi: Estimated sources ◦ Goal: Yi=Si BSS & ICA Speech Recognition - Spring 2008 3

Herault and Jutten (HJ) Algorithm Presented in GRETSI’ 85, COGNITAVA’ 85 and Snowbird’ 86 Choosing m 12 and m 21 correctly results in separation: BSS & ICA Speech Recognition - Spring 2008 4

Herault and Jutten (HJ) Algorithm Main Idea: The algorithm: Independence (ICA) BSS & ICA Speech Recognition - Spring 2008 5

Types of Mixtures Linear Non. Linear UnderOver determined Memory ◦ Instantaneous ◦ Convolutive BSS & ICA Speech Recognition - Spring 2008 6

Illustration of ICA with 2 signals (Geometric Method) s 2 x 2 a 1 a 2 a 1 s 1 Original s x 1 Mixed signals BSS & ICA Speech Recognition - Spring 2008 7

Illustration of ICA with 2 signals (Geometric Method) s 2 x 2 a 1 a 2 a 1 s 1 Original s x 1 Mixed signals Step 1: Sphering BSS & ICA Speech Recognition - Spring 2008 Step 2: Rotatation 8

Excluded case There is one case when rotation doesn’t matter. This case cannot be solved by basic ICA. …when both densities are Gaussian BSS & ICA Speech Recognition - Spring 2008 9

Geometric Method Combination= BSS & ICA Speech Recognition - Spring 2008 10

ICA Methods: ICA method = Objective function + Optimization algorithm ● Objective Function: ● Mutual Information ● ● If Ui are independent from each other then i(pu)=0 Moments and cumulants … Algorithm : minimizes/maximizes function: ● ● Gradient-Based … BSS & ICA Speech Recognition - Spring 2008 11

Applications Speech Processing: ◦ ◦ Noise Cancelation in car environment As a preprocess in speech recognition systems Speech enhancement in Reverberant environment Cocktail party problem Other: ◦ Image Denoising ◦ Economic time series ◦ Brain signals (EEG and MEG) BSS & ICA Speech Recognition - Spring 2008 12

Example of Separation Mixture 1 Mixture 2 Estimated Source 1 Estimated Source 2 Ref: T. -W. Lee, A. J. Bell, and R. Lambert. Blind separation of delayed and convolved sources. In Advances in Neural Information Processing Systems, Volume 9, pages 758764. MIT Press, 1997. BSS & ICA Speech Recognition - Spring 2008 13

Questions? BSS & ICA Speech Recognition - Spring 2008 14