Linear Discriminant Analysis Linear Discriminant Analysis n n

Linear Discriminant Analysis

Linear Discriminant Analysis n n n Why n To identify variables into one of two or more mutually exclusive and exhaustive categories. n To examine whether significant differences exist among the groups in terms of the predictor variables. What n The analysis helps determine what predictor variables contribute most to intergroup differences. n It then classifies cases to one of the groups based on the values of the predictor variables. How n Using a combination of MANOVA, PCA and MLP.

LDA n Assumptions n Absence of outliers n Equal samples size n Many data n Homogeneity of variance-covariance n Linear relationship n No multicolinearity

LDA n Toy example IVs DVs =X

LDA n First step: Significance testing of the overall classifier in order to know if a set of discriminant functions can significantly predict group membership or not n Second step: Significance testing for each discriminant function. n Third step: Computation of the (standardized, unstandardized) discriminant functions

LDA - Overall Testing n Sum of Square and Cross Product SSCP=

LDA - Overall Testing n Canonical Correlation Matrix n Error and hypothesis matrices

LDA - Overall Testing n n Computing W (WLR) where s = min(df, q), lk is kth eigenvalue extracted from Hi. E-1 and |E| (as well as |E+Hi|) is the determinant. The overall test is significant

LDA - Individual Testing n Eigenvalues and eigenvectors decomposition of the matrix: E-1 H= PCA E-1 H

LDA - Individual Testing n Canonical Discriminant Analysis Squared canonical correlation (Can also obtained from the eigenvalues of the correlation matrix R) Canonical correlation

LDA - Individual Testing n Significance test for the canonical correlations n A significant output indicates that there is a variance share between IV and DV sets Procedure: n n We test for all the variables (m=1, …, min(p, q)) If significant, we removed the first variable (canonical correlate) and test for the remaining ones (m=2, …, min(p, q) Repeat

LDA - Individual Testing n Significance test for the canonical correlations Since all canonical variables are significant, we will keep them all.

LDA – Projection of the solution P=VY Second discriminant function First group Second group Third group First discriminant function

LDA – Discriminant Functions D 1 D 2 D 3 b 0 b 1 b 2 b 3 b 4 n Class membership is given by: Max(D 1, D 2, D 3) n Example x=(86, 6, 35, 6. 5); n n n D 1= 122. 817 (MAX) D 2= 103. 706 D 3= 105. 642