Principal Components ShyhKang Jeng Department of Electrical Engineering

Principal Components Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and Multimedia 1

Concept of Principal Components x 2 x 1 2

Principal Component Analysis Explain the variance-covariance structure of a set of variables through a few linear combinations of these variables Objectives – Data reduction – Interpretation Does not need normality assumption in general 3

Principal Components 4

Result 8. 1 5

Proof of Result 8. 1 6

Result 8. 2 7

Proof of Result 8. 2 8

Proportion of Total Variance due to the kth Principal Component 9

Result 8. 3 10

Proof of Result 8. 3 11

Example 8. 1 12

Example 8. 1 13

Example 8. 1 14

Geometrical Interpretation 15

Geometric Interpretation 16

Standardized Variables 17

Result 8. 4 18

Proportion of Total Variance due to the kth Principal Component 19

Example 8. 2 20

Example 8. 2 21

Principal Components for Diagonal Covariance Matrix 22

Principal Components for a Special Covariance Matrix 23

Principal Components for a Special Covariance Matrix 24

Sample Principal Components 25

Sample Principal Components 26

Example 8. 3 27

Example 8. 3 28

Scree Plot to Determine Number of Principal Components 29

Example 8. 4: Pained Turtles 30

Example 8. 4 31

Example 8. 4: Scree Plot 32

Example 8. 4: Principal Component One dominant principal component – Explains 96% of the total variance Interpretation 33

Geometric Interpretation 34

Standardized Variables 35

Principal Components 36

Proportion of Total Variance due to the kth Principal Component 37

Example 8. 5: Stocks Data Weekly rates of return for five stocks – X 1: – X 2: – X 3: – X 4: – X 5: Allied Chemical du Pont Union Carbide Exxon Texaco 38

Example 8. 5 39

Example 8. 5 40

Example 8. 6 Body weight (in grams) for n=150 female mice were obtained after the birth of their first 4 litters 41

Example 8. 6 42

Comment An unusually small value for the last eigenvalue from either the sample covariance or correlation matrix can indicate an unnoticed linear dependency of the data set One or more of the variables is redundant and should be deleted Example: x 4 = x 1 + x 2 + x 3 43

Check Normality and Suspect Observations Construct scatter diagram for pairs of the first few principal components Make Q-Q plots from the sample values generated by each principal component Construct scatter diagram and Q-Q plots for the last few principal components 44

Example 8. 7: Turtle Data 45

Example 8. 7 46

Large Sample Distribution for Eigenvalues and Eigenvectors 47

Confidence Interval for li 48

Approximate Distribution of Estimated Eigenvectors 49

Example 8. 8 50

Testing for Equal Correlation 51

Example 8. 9 52

Monitoring Stable Process: Part 1 53

Example 8. 10 Police Department Data *First two sample cmponents explain 82% of the total variance 54

Example 8. 10: Principal Components 55

Example 8. 10: 95% Control Ellipse 56

Monitoring Stable Process: Part 2 57

Example 8. 11 T 2 Chart for Unexplained Data 58

Example 8. 12 Control Ellipse for Future Values *Example 8. 10 data after dropping out-of-control case 59

Example 8. 12 99% Prediction Ellipse 60

Avoiding Computation with Small Eigenvalues 61