Canonical Correlation Analysis ShyhKang Jeng Department of Electrical

Canonical Correlation Analysis Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and Multimedia 1

Canonical Correlation Analysis Seeks to identify and quantify the association between two sets of variables Examples – Relating arithmetic speed and arithmetic power to reading speed and reading power – Relating government policy variables with economic goal variables – Relating college “performance” variables with precollege “achievement” variables 2

Canonical Correlation Analysis Focuses on the correlation between a linear combination of the variables in one set and a linear combination of the variables in another set First to determine the pair of linear combinations having the largest correlation Next to determine the pair of linear combinations having the largest correlation among all pairs uncorrelated with the initially selected pair, and so on 3

Canonical Correlation Analysis Canonical variables – Pairs of linear combinations used in canonical correlation analysis Canonical correlations – Correlations between the canonical variables – Measures the strength of association between the two sets of variables Maximization aspect – Attempt to concentrate a high-dimensional relationship between two sets of variables into a few pairs of canonical variables 4

Example 10. 5 Job Satisfaction 5

Example 10. 5 Job Satisfaction 6

Canonical Variables and Canonical Correlations 7

Canonical Variables and Canonical Correlations Covariances between pairs of variables from different sets are contained in S 12 or, equivalently S 21 When p and q are relatively large, interpreting the elements of S 12 collectively is very difficult Canonical correlation analysis can summarize the associations between two sets in terms of a few carefully chosen covariances rather than the pq covariances in S 12 8

Canonical Variables and Canonical Correlations 9

Canonical Variables and Canonical Correlations First pair of canonical variables – Pair of linear combinations U 1, V 1 having unit variances, which maximize the correlation kth pair of canonical variables – Pair of linear combinations Uk, Vk having unit variances, which maximize the correlation among all choices uncorrelated with the previous k-1 canonical variable pairs 10

Result 10. 1 11

Result 10. 1 12

Result 10. 1 13

Proof of Result 10. 1 14

Proof of Result 10. 1 15

Proof of Result 10. 1 16

Proof of Result 10. 1 17

Proof of Result 10. 1 18

Canonical Variates 19

Comment 20

Comment 21

Example 10. 1 22

Example 10. 1 23

Example 10. 1 24

Alternative Approach 25

Identifying Canonical Variables by Correlation 26

Example 10. 2 27

Canonical Correlations vs. Other Correlation Coefficients 28

Example 10. 3 29

Sample Canonical Variates and Sample Canonical Correlations 30

Result 10. 2 31

Matrix Forms 32

Sample Canonical Variates for Standardized Observations 33

Example 10. 4 34

Example 10. 5 Job Satisfaction 35

Example 10. 5 Job Satisfaction 36

Example 10. 5: Sample Correlation Matrix Based on 784 Responses 37

Example 10. 5: Canonical Variate Coefficients 38

Example 10. 5: Sample Correlations between Original and Canonical Variables 39

Matrices of Errors of Approximations 40

Matrices of Errors of Approximations 41

Matrices of Errors of Approximations 42

Example 10. 6 43

Example 10. 6 44

Example 10. 6 45

Sample Correlation Matrices between Canonical and Component Variables 46

Proportion of Sample Variances Explained by the Canonical Variables 47

Proportion of Sample Variances Explained by the Canonical Variables 48

Example 10. 7 49

Result 10. 3 50

Bartlett’s Modification 51

Test of Significance of Individual Canonical Correlations 52

Example 10. 8 53

Example 10. 8 54