Applied Multivariate Quantitative Methods Factor Analysis By Jenpei

Applied Multivariate Quantitative Methods Factor Analysis By Jen-pei Liu, Ph. D Division of Biometry, Department of Agronomy, National Taiwan University and Wei-Chie, MD, Ph. D Department of Public Health National Taiwan University 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 1

Factor Analysis n n Introduction Exploratory Factor Analysis n n n Confirmatory Factor Analysis and LISREL n n n Principal Components Factor Analysis Maximum Likelihood Method Other Methods Examples Procedures Examples Summary 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 2

Introduction Correlation between Test Scores Classics French Classics 1. 00 0. 83 French 0. 83 1. 00 English 0. 78 0. 67 Math 0. 70 0. 67 Do. P 0. 66 0. 65 Music -0. 63 0. 57 English Math Do. P 0. 78 0. 70 0. 66 0. 67 0. 65 1. 00 0. 64 0. 54 0. 64 1. 00 0. 45 0. 54 0. 45 1. 00 0. 51 0. 40 Music 0. 63 0. 57 0. 51 0. 40 1. 00 ch Do. P: Discrimination of Pitch; Source: Manly (2005) and Spearman (1904) 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 3

Introduction Correlation between Test Scores Classics French Classics 1. 00 0. 83 French 0. 83 1. 00 English 0. 78 0. 67 Math 0. 70 0. 67 Do. P 0. 66 0. 65 Music 0. 63 0. 57 English Math Do. P 0. 78 0. 70 0. 66 0. 67 0. 65 1. 00 0. 64 0. 54 0. 64 1. 00 0. 45 0. 54 0. 45 1. 00 0. 51 0. 40 Music 0. 63 0. 57 0. 51 0. 40 1. 00 ch Do. P: Discrimination of Pitch; Source: Manly (2005) and Spearman (1904) 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 4

Introduction n Structure and property of the correlation (covariance matrix) matrix Any two rows are almost proportional if the diagonals are ignored For classics (1) and English (3) 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 5

Introduction n n A constant ratio between the rows of the correlation matrix indicates that there might be a model for the variation of the data The six standardized test scores can be described as Xi = ai. F + ei, i = 1, …, 6 where Xi is the ith standardized test score, ai is a constant, F is a “factor” value that has mean of 0 and variance of 1, and ei is the part of Xi that is specific to the ith test only 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 6

Introduction If F and ei are assumed independent n Var(Xi)=Var(ai. F + ei) = Var(ai. F) + Var(ei) = ai 2 Var(F) + Var(ei) = ai 2 + Var(ei) n Since Xi is a standardized score, 1 = ai 2 + Var(ei) ai is called the factor loading ai 2: proportion of variance of Xi that is accounted for by factor F 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 7

Introduction n One-factor Model 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 8

Introduction n Examples n Education n Indicators n n Factors n n n 12/12/2021 test scores on Classics, French, English, Mathematics, Discrimination of Pitch, and Music Intelligence study attitude … Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 9

Introduction n Examples n Quality of Life n Indicators n n Factors (domain) n n 12/12/2021 Scores from 1 to 5 for each of questions in the questionnaires Physical Mental Social … Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 10

Introduction n Examples n Hospital Anxiety and Depression Scale n Indicators n n Factors (construct) n n n 12/12/2021 Scores from each of 14 questions in the questionnaires Anxiety Depression … Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 11

Introduction n Two-factor model 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 12

Introduction n Objectives of factor analysis n n n To reduce the number of variables (indictors) to the smallest number of common factors To identify the common factors (latent constructs) to best explain the intercorrelation among indicators and to build the most parsimonious factor model Principal component analysis also tries to reduce the number of variables. However, factor analysis is based on a model 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 13

Introduction Example: Four correlated variables: X 1, X 2, X 3, X 4 X 1=X 1 X 2=X 2 X 3=2 X 1+3 X 2 X 4=5 X 1+4 X 2 n X 3 and X 4 are linear function of X 1 and X 2 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 14

Introduction n If there are two independent variables Y 1 and Y 2, X 1=Y 1, X 3=2 Y 1+3 Y 2 X 2=Y 2, X 4=5 Y 1+4 Y 2 Y 1 and Y 2 are common factors 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 15

Introduction n m-factor model 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 16

Introduction n n Xi is the ith score with mean 0 and unit variance ai 1 to aim are the factor loadings for the ith test F 1 to Fm are m uncorrelated common factors, each with mean 0 and unit variance ei is a factor specific only to the ith test that is uncorrelated with any of the common factors and has zero mean 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 17

Introduction 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 18

Introduction 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 19

Introduction 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 20

Introduction 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 21

Introduction 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 22

Introduction n Exploratory factor analysis (EFA) n Little or no knowledge about the factor structure n n The number of factors The number of indicators for each factor Which indicators represent which factors To collect data and explore or search for a factor structure or theory which can explain the correlation among indicators 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 23

Introduction n Confirmatory factor analysis (CFA) n Factor structure is known or hypothesized (specified) a priori n n n Complete factor structure with their respective indicators Nature pattern of factor loadings To empirically verify or confirm the factor structure 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 24

Exploratory Factor Analysis Principal Components Factor (PCF) Analysis p variables X 1, X 2, …, Xp, and p principal components Z 1, Z 2, …, Zp Z 1=b 11 X 1+b 12 X 2+…+b 1 p. Xp Z 2=b 21 X 1+b 22 X 2+…+b 2 p. Xp. . Zp=bp 1 X 1+bp 2 X 2+…+bpp. Xp n 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 25

Exploratory Factor Analysis 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 26

Exploratory Factor Analysis n The inverse relationship between X and Z X 1=b 11 Z 1+b 21 Z 2+…+bp 1 Zp X 2=b 12 Z 1+b 22 Z 2+…+bp 2 Zp. . Xp=b 1 p. X 1+b 2 p. Z 2+…+bpp. Zp 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 27

Exploratory Factor Analysis n For factor analysis, only m of the principal components are retained X 1=b 11 Z 1+b 21 Z 2+…+bm 1 Zm + e 1 X 2=b 12 Z 1+b 22 Z 2+…+bm 2 Zm + e 2. . Xp=b 1 p. X 1+b 2 p. Z 2+…+bmp. Zm + ep 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 28

Exploratory Factor Analysis n n For factor analysis, factors have zero mean and unit variance. Zi is divided by its standard deviation i, the square root of the corresponding eigenvalue X 1= 1 b 11 F 1+ 2 b 21 F 2+…+ mbm 1 Fm + e 1 X 2= 1 b 12 F 1+ 2 b 22 F 2+…+ mbm 2 Fm + e 2. . Xp= 1 b 1 p. F 1+ 2 b 2 p. F 2+…+ mbmp. Fm + ep 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 29

Exploratory Factor Analysis n The unrotated factor model is X 1= a 11 F 1+a 12 F 2+…+a 1 m. Fm + e 1 X 2= a 21 F 1+a 22 F 2+…+a 2 m. Fm + e 2. . Xp= ap 1 F 1+ap 2 F 2+…+apm. Fm + ep where Fi = Zi/ i and aij= jbji 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 30

Exploratory Factor Analysis n n Varimax Rotation that produces the maximum variation n Objectives n n n 12/12/2021 To have a factor structure in which each variables loads highly on one and only one factor A given variable should have a high loading on one factor and near zero loadings on others Each factor represents a distinct construct Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 31

Exploratory Factor Analysis n Principal Axis Factor Analysis (PAF): an iterative PCF method n Step 1: Assume that the prior estimates of the communalities are one. A PCF is obtained. Based on the number of factors retained, estimates of factor loadings are obtained and then are used to re-estimate the communalities. 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 32

Exploratory Factor Analysis n Principal Axis Factor (PAF) Analysis: an iterative PCF method n n Step 2: Compute the maximum change in estimated communalities among all indicators between two iterations. Step 3: If the maximum change in communalities is greater than a pre-specified convergence criterion, then replace the diagonal elements of the original correlation matrix by the estimated communalities 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 33

Exploratory Factor Analysis n Principal Axis Factor (PAF) Analysis: an iterative PCF method n Step 4: A new principal components factor analysis is performed using the modified correlation matrix and Step 2 is repeated. Step 2 to Step 4 are repeated until the maximum change in the communalities is smaller than the pre-specified convergence criterion 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 34

Exploratory Factor Analysis n Factor Scores F* = XG(G’G)-1 where F* is the nxm matrix of factor scores, with one row for each of the n rows of data, X is the nxp matrix of original data, G is the pxm matrix of factor loadings 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 35

Exploratory Factor Analysis n Maximum Likelihood (ML) Method n n n Assumption: multivariate normal Cov(X) = = A’A + W Maximize the log-likelihood function Ln(l)=-(n/2){ln(|A’A + W|+tr(A’A + W)-1 C}+k Equivalent to minimize f = ln(|A’A + W|+tr(A’A + W)-1 C 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 36

Exploratory Factor Analysis 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 37

Exploratory Factor Analysis For a given W, find the solutions to CW-1 A=A(I+A’W-1 A) n n Let Ao be the solution, a solution for W is given as W = diag(C- Ao A’o) 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 38

Exploratory Factor Analysis n Selection of The Number of Factors n n Cumulative proportion of variation of the r common factors > 80% or 90% When the eigenvalue of the rth common factor > average variation. When the correlation matrix is used, the average variation is 1. Choose the r common factors which eigenvalues are greater than 1 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 39

Exploratory Factor Analysis n Selection of The Number of Factors n n Decree plot: number of factors vs. eigenvalues – choose the number of factors when the curve becomes flat Maximum likelihood method: Test the hypothesis until failure to reject the null hypothesis Ho: = A’A + W vs. Ha: A’A + W 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 40

Exploratory Factor Analysis n Selection of The Number of Factors Test Statistic: M = kln[|A’A+W|/|R|], where k = n-2(p+4 r+11)/6. Reject Ho if M > 2 , df=[(p-r)2 -p-r]/2, r = the number of factors 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 41

Exploratory Factor Analysis n Selection of The Number of Factors Step 1: Start r=2, if fail to reject Ho, stop and select 2 as the number of factors, Step 2: otherwise let r=3, re-compute A’A+W and perform the test again. If fail to reject Ho, stop and select 3 as the number of factors Step 3: Repeat Step 1 and Step 2 until failing to reject the null hypothesis 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 42

Exploratory Factor Analysis n When to rotate n n Find the factor loadings >= 0. 5 (ignore the sign) The large and moderate factor loadings indicate relatively good correlation between indicators and factors Try to avoid the situation where a large number of indicators is related strongly to only a few factors Rotate the factors such that there is no overlapping of large or moderate factor loadings among factors 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 43

Exploratory Factor Analysis n Examples (PCF) Evaluation of two coking methods for fish: Shen (1998) and Rencher (1995) Evaluation items (score: 1 -10) aroma (X 1) taste (X 2) texture (X 3) moisture (X 4) 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 44

Exploratory Factor Analysis Examples (PCF) Evaluation of two coking methods for fish Correlation Matrix(n=20) X 1 X 2 X 3 X 4 X 1 1 0. 62938 0. 43116 0. 33543 X 2 1 0. 22779 0. 30179 X 3 1 0. 75061 X 4 1 n 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 45

Exploratory Factor Analysis n Examples (PCF) Evaluation of Two coking methods for fish Eigenvalues and 1 2 2. 34486 1. 04840 0. 50584 0. 44079 0. 44267 0. 60513 0. 52726 -0. 47040 0. 51978 -0. 46717 12/12/2021 Eigenvectors 3 4 0. 40394 0. 20280 -0. 64930 -0. 35810 0. 58061 0. 31742 -0. 26945 0. 65432 0. 41073 -0. 58556 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 46

Exploratory Factor Analysis n Examples (PCF)-Continued The first 2 principal components accounts for 84. 83% of total variation. We take 2 factors Computation of factor loading for the first factor and X 3 a 31 = ( 1)b 13 = ( 2. 34486)(0. 52726) = 0. 80738 h 12 = a 112+a 122 = (0. 77459)2 + (0. 45133)2 = 0. 80370 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 47

Exploratory Factor Analysis n Examples (PCF) Evaluation of Two coking methods for fish Factor Loadings and Communality Variable X 1 X 2 X 3 X 4 Variance Proportion Cum. Prop. 12/12/2021 F 1 0. 77459 0. 67786 0. 80738 0. 79593 2. 34486 0. 5862 F 2 0. 45133 0. 61980 -0. 48164 -0. 47834 1. 04840 0. 2621 0. 8483 Communality 0. 80370 0. 84340 0. 88385 0. 86232 3. 39327 0. 8483 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D Specific Variance 0. 19630 0. 15660 0. 11615 0. 13768 48

Exploratory Factor Analysis n Examples (PCF)-Continued n n The communalities are quite high (> 0. 80) for all variables. Most of the variation for the 4 variables can be accounted by the 2 common factors The factor loadings for the unrotated first factor are very large and have the same sign. The first factor seems to represent the overall quality of the cooking method 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 49

Exploratory Factor Analysis n Examples (PCF)-Continued n The absolute values of factor loadings for the unrotated second factor are in the same magnitude. However, aroma and taste have different sign with texture and moisture. This factor is a contrast between aroma with taste and texture with moisture. n However, the factor loadings within each factor have the same magnitudes. Therefore, some of the 4 variables are strongly related to the two factors – a undesirable properties of the factor. We need to see. Copyright whether a. Liu, rotation can help. by Jen-pei Ph. D and 12/12/2021 Wei-Chu Chie, MD, Ph. D 50

Exploratory Factor Analysis Examples (PCF)-Continued Unrotated Varimax Rotation F 1 F 2 0. 77449 0. 45133 0. 27554 0. 85310 0. 67786 0. 61960 0. 09120 0. 91383 0. 80738 -0. 48164 0. 92269 0. 18025 0. 79593 -0. 47834 0. 91196 0. 17507 n 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 51

Exploratory Factor Analysis n Examples (PCF)-Continued n n n It can be verified that the communalities are unchanged based on the factor loading after rotation Factor loadings greater than 0. 5 (ignore the sign) are underlined. These large and moderate loadings indicate how the variables are related to the factors Factor 1 composes of texture and moisture and factor 2 consists of aroma and taste. 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 52

Exploratory Factor Analysis n Estimation of Correlation Matrix 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 53

Exploratory Factor Analysis n Example (PCF) – European Employment (n=30) Un-rotated Indicators F 1 F 2 F 3 AGR 0. 90 -0. 03 -0. 34 MIN 0. 66 0. 00 0. 63 MAN -0. 43 0. 58 -0. 61 PS -0. 56 0. 15 -0. 36 CON -0. 39 -0. 33 0. 09 SER -0. 67 -0. 55 0. 08 FIN -0. 23 -0. 74 -0. 12 SPS -0. 76 0. 07 0. 44 TC -0. 36 0. 69 0. 50 12/12/2021 Varimax Rotation F 4 F 1 F 2 F 3 F 4 0. 02 0. 85 0. 10 0. 27 -0. 36 0. 12 0. 11 0. 30 0. 86 -0. 10 0. 06 -0. 03 0. 32 -0. 89 -0. 09 0. 02 -0. 19 -0. 04 -0. 64 0. 14 0. 81 -0. 02 0. 08 -0. 04 0. 95 0. 17 -0. 35 -0. 48 -0. 15 0. 65 -0. 50 -0. 08 -0. 98 0. 00 -0. 01 -0. 33 -0. 91 -0. 17 -0. 12 0. 04 -0. 73 0. 57 -0. 03 -0. 14 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 54

Exploratory Factor Analysis n Example (PCF) – European Employment n n Factor 1: +high loading in AGR, -high loadings in SPS amd TC: rural industry rather than social service and communicaton Factor 2: +moderate loading in TC and – high loading in FIN: lack of finance industries n 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 55

Exploratory Factor Analysis n Example (PCF) – European Employment n n Factor 3: +high loading in MIN, -high loadings in MAN –moderate loading in PS: mining rather than manufacturing Factor 4: +high loading in CON and +moderate high in SER: construction and service industry 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 56

Exploratory Factor Analysis n Example: Test scores (PAF) M P C E H F Correlation Matrix(n=200) M P C E H F 1. 00 0. 62 0. 54 0. 32 0. 284 0. 37 1. 00 0. 51 0. 38 0. 351 0. 43 1. 00 0. 36 0. 336 0. 405 1. 00 0. 686 0. 730 1. 00 0. 7345 1. 00 M: mathematics, P: physics, C: chemistry E: English, H: history, F: French; source: Sharma (1996) 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 57

Exploratory Factor Analysis n Example (PAF): Test scores History of PAF Iter. Change 1. 359 2. 128 3. 042. . 9. 001 12/12/2021 analysis M P. 766. 714. 698. 626. 679. 598 C E H F. 641. 797. 812. 829. 513. 725. 744. 784. 471. 698. 719. 774 . 677. 581 . 450. 680. 697. 783 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 58

Exploratory Factor Analysis n Example (PAF): Test scores Cum. Indicator Eigenvalue Prop. Factor 1 M 3. 0281 0. 7826 0. 63584 P 0. 8410 1. 0000 0. 65784 C 0. 0004 1. 0004 0. 59812 E 0. 0003 1. 0007 0. 76233 H -0. 0003 1. 0004 0. 74908 F -0. 0004 1. 0000 0. 83129 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D Factor 2 0. 52255 0. 38549 0. 30447 -0. 31509 -0. 36797 -0. 30329 59

Exploratory Factor Analysis n Example (PAF): Test scores Unrotated Varimax Factor 1 Factor 2 Communality M 0. 63584 0. 52255 0. 15200 0. 80886 0. 677354 P 0. 65784 0. 38549 0. 25687 0. 71790 0. 581356 C 0. 59812 0. 30447 0. 26309 0. 61744 0. 450447 E 0. 76233 -0. 31509 0. 78676 0. 24786 0. 680426 H 0. 74908 -0. 36797 0. 81055 0. 19881 0. 696517 F 0. 83129 -0. 30329 0. 83205 0. 30118 0. 783020 Variance 3. 028093 0. 841027 2. 126592 1. 742525 3. 86912 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 60

Exploratory Factor Analysis n Example (ML): Meteorological Measurements n X 1: daily high temperature (F) n X 2: daily low temperature (F) n X 3: daily soil high temperature (F) n X 4: daily soil low temperature (F) n X 5: relative humidity for daily high temperature n X 6: relative humidity for daily low temperature n X 7: daily wind speed n X 8: daily amount of vaporization 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 61

Exploratory Factor Analysis n Example (ML): Meteorological Measurements (n=20) Correlation matrix X 1 X 2 X 3 X 4 X 5 X 1 1 0. 737 0. 931 0. 446 -0. 479 X 2 1 0. 827 0. 865 -0. 255 X 3 1 0. 616 -0. 381 X 4 1 -0. 154 X 5 1 X 6 X 7 X 8 Source: Rencher (1995) and Shen (1998) 12/12/2021 X 6 -0. 551 -0. 019 -0. 341 0. 382 0. 500 1 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D X 7 0. 384 0. 536 0. 466 0. 683 -0. 106 0. 299 1 X 8 0. 879 0. 597 0. 848 0. 375 -0. 539 -0. 568 0. 494 1 62

Exploratory Factor Analysis n Example (ML): Meteorological Measurements n Ho: 2 factors are sufficient Ha: More factors are needed Test Statistic M=[n-2(p+4 r 11)/6] ln[|A’A+W|/|R|] = 24. 789 degrees of freedom=[(p-r)2 -p-r]/2 =[(8 -2)2 -8 -2]/2 = 26/2=13 M=24. 789 > 20. 05, 13=22. 362 Reject Ho at the 5% significance level. Two factors are not sufficient. Test whether 3 will be sufficient 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 63

Exploratory Factor Analysis n Example (ML): Meteorological Measurements n Ho: 3 factors are sufficient Ha: More factors are needed Test Statistic M=[n-2(p+4 r 11)/6] ln[|A’A+W|/|R|] = 12. 186 degrees of freedom=[(p-r)2 -p-r]/2 =[(8 -3)2 -8 -3]/2 = 14/2=7 M=12. 186 < 20. 05, 7=14. 076 Fail to reject Ho at the 5% significance level. Select three factors 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 64

Exploratory Factor Analysis n Example (ML): Meteorological Measurements Un-rotated Factor Loadings Indicator F 1 X 1 0. 3843 X 2 0. 5363 X 3 0. 4658 X 4 0. 6827 X 5 -0. 1063 X 6 0. 2990 X 7 1. 0000 X 8 0. 4942 12/12/2021 F 2 0. 8937 0. 6462 0. 8362 0. 2868 -0. 4919 -0. 6779 -0. 0000 0. 7564 F 3 Communality -0. 0972 0. 9558 -0. 4964 0. 9503 0. 0989 0. 9259 0. 6171 0. 9291 0. 2275 0. 3050 0. 6104 0. 9215 -0. 0000 1. 0000 -0. 3091 0. 9119 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 65

Exploratory Factor Analysis n Example (ML): Meteorological Measurements Factor Loadings by Varimax Indicator F 1 X 1 0. 8223 X 2 0. 2980 X 3 0. 6717 X 4 -0. 0554 X 5 -0. 5416 X 6 -0. 8788 X 7 0. 0728 X 8 0. 8343 12/12/2021 F 2 0. 5062 0. 9058 0. 6617 0. 8759 -0. 1034 0. 2523 0. 3602 0. 3103 F 3 Communality 0. 1528 0. 9558 0. 2025 0. 9503 0. 1920 0. 9259 0. 3992 0. 9291 -0. 0318 0. 3050 0. 2925 0. 9215 0. 9300 1. 0000 0. 3459 0. 9119 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 66

Exploratory Factor Analysis n Differences between principal components analysis and factor analysis n n PCA emphasize on explaining the variation in the data Factor analysis is to explain the correlation among indicators For PCA, indicators form an index such as Comsumer Price Index or Dow Jones Industrial Average For factor analysis, indicators reflect the presence of unobservable constructs or factors 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 67

Exploratory Factor Analysis n Factor Indeterminacy n The factor analysis solution is not unique n Due to factor rotation problem n n Due to estimation of communality n n Infinite ways to rotate the factors Different methods for estimation of communality provide different solutions Factor analysis is an 12/12/2021 ART than a science Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 68

Exploratory Factor Analysis n Are the data appropriate for factor analysis? n n Examine the correlation. Pattern of groups of variables with high correlations Partial correlations after controlling all other variables. Low partial correlations Kaiser-Meyer-Olkin (KMO) measure for sampling adequacy > 0. 6 The overall square root of mean square residuals (RMSR) < 0. 01 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 69

Exploratory Factor Analysis n Kaiser-Meyer-Olkin (KMO) measure for sampling adequacy KMO measure Recommendation n n n >= 0. 90 0. 80+ 0. 70+ 0. 6+ 0. 5+ Below 0. 5 12/12/2021 Marvelous Meritoroius Middling Mediocre Miserable Unacceptable Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D desirable tolerable 70

Exploratory Factor Analysis n The overall square root of mean square residuals (RMSR) n n n Reproduced correlation matrix = A’A Residual correlation matrix = R – A’A RMSR is the square root of the average squared values of off-diagonal entries of the residual correlation matrix 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 71

Exploratory Factor Analysis n Principal Axis Factor (PAF) Analysis: an iterative PCF method SAS default: 30 iterations with a convergence criterion = 0. 001. If more iterations are required, the data may not be suitable for factor analysis 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 72

Exploratory Factor Analysis n n Factor loadings are not unique The variance in common between the factor and the indicators is not unique either The variance in common between the factor and the indicators is not a meaningful measure of factor importance However, the objective of factor analysis is to explain the intercorrelations among indicators and is not to account for the total variation in the data 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 73

Exploratory Factor Analysis n n n PCF analysis labeled the first few principal components accounting for most of variation as common factors and the other principle components as the specific part. PCF analysis can not really be called factor analysis PAF implicitly assumes a factor model and it is a preferred method by most of researchers 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 74

Confirmatory Factor Analysis n Correlation vs. Covariance Matrix? n n n The objective of exploratory factor analysis (EFA) is to investigate the intercorrelations among indicators The correlation matrix is always exclusively used in the EFA The correlation is the covariance matrix of standardized variables 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 75

Confirmatory Factor Analysis n Correlation vs. Covariance Matrix? n n The standardized variables are unitless and hence are not scale invariant Most of confirmatory factor models are scale invariant The maximum likelihood procedure for CFA are derived from covariance matrix One should always use the covariance matrix for confirmatory factor analysis 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 76

Confirmatory Factor Analysis n Objectives n n Given the covariance matrix, to estimate the parameters of the hypothesized factor model To determine the fit of the hypothesized factor model. 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 77

Confirmatory Factor Analysis n The Models 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 78

Confirmatory Factor Analysis 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 79

Confirmatory Factor Analysis 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 80

Confirmatory Factor Analysis n n is a pxp matrix of variances and covariances of indicators A is a pxm matrix of factor loadings is a mxm matrix of the variances and covariances of latent construct W us a pxp matrix of the variancs and covariances of the error terms 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 81

Confirmatory Factor Analysis n Confirmatory factor analysis is one of linear structural relation (LISREL) Parameter Matrix Order A pxm mxm W pxp 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 82

Confirmatory Factor Analysis n The parameters of the models n n n Free parameters: the parameters to be estimated Fixed parameters: the parameters not be estimated and values to be provided Constrained parameters: the parameters with values constrained to to be equal to other free parameters 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 83

Confirmatory Factor Analysis n The parameters of the models n n n Constrained parameters: the variances of the errors of all indicators are constrained to be equal Most of the latent construct such as attitudes, intelligence, or excellence do not have a natural measurement scale. We need to define the metric or scale for the latent construct In general, the scale of the latent construct is defined such as it is the same as one of indicators used to measure construct 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 84

Confirmatory Factor Analysis n Estimation of the parameters of the hypothesized factor model n Example: one-factor model 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 85

Confirmatory Factor Analysis n Estimation of the parameters of the hypothesized factor model n Example: one-factor model: assume that p=2 and variance of F is 1 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 86

Confirmatory Factor Analysis n Estimation of the parameters of the hypothesized factor model n Example: one-factor model: assume that p=3 and variance of F is 1 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 87

Confirmatory Factor Analysis n Estimation of the parameters of the hypothesized factor model n Example: one-factor model: assume that p=4 and variance of F is 1 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 88

Confirmatory Factor Analysis n Estimation of the parameters of the hypothesized factor model n n The number of parameters > the number of equations: unidentified model - one-factor model with p=2 set w 12 = w 22 = w three equations for three parameters The number of parameters = the number of equations: justidentified model - one-factor model with p=3 The number of parameters > the number of equations: overidentified model - one-factor model with p=4 Degrees of freedom = over-identifying equations = p(p+1)/2 - # of free parameters: p=4: df = 4(4+1)/2 – 8 =2 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 89

Confirmatory Factor Analysis n Estimation of the parameters of the hypothesized factor model n n Maximum likelihood estimation procedure for all free and constrained parameters Convergence Statistical significance Inadmissible estimates n n n 12/12/2021 Factoring loadings outside [-1, 1] Negative variances of the construct and error terms Variances of the error terms are greater than 1 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 90

Confirmatory Factor Analysis n n Evaluation of Model Fit Ho: = ( ) vs. Ho: ( ) The 2 test Test Statistic: M = ln[| |/|C|]+Tr(C ) - p where is the estimated covariance matrix estimated using MLE under the assumed factor model. Reject Ho if M > 2 , df=[p(p+1)/2 - # of free parameters], 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 91

Confirmatory Factor Analysis n Evaluation of Model Fit n Other measures 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 92

Confirmatory Factor Analysis n Evaluation of Model Fit n Other measures 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 93

Confirmatory Factor Analysis n Evaluation of Model Fit n Other measures 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 94

Confirmatory Factor Analysis n Evaluation of Model Fit n Other measures 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 95

Confirmatory Factor Analysis n Evaluation of Model Fit n Other measures n n n 12/12/2021 The residual matrix: C contains the variances and covariance that can not be explained by the assumed factor model. No more than 5% of the standardized residuals should be greater than 1. 96 Squared multiple correlation between the indicators and construct– estimated variances of construct (estimated communalities) Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 96

Confirmatory Factor Analysis n Evaluation of Model Fit n Other measures 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 97

Confirmatory Factor Analysis n Alternative measures of fit 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 98

Confirmatory Factor Analysis n Alternative Measures of Fit 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 99

Confirmatory Factor Analysis n Evaluation Criteria Fit indices Chi-square/df GFI AGFI NCP MDN 12/12/2021 Criteria NS and as small as possible <3 >0. 9 >0. 8 >0. 9 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 100

Confirmatory Factor Analysis n Evaluation Criteria Fit indices TLI RNI NFI NNFI RMSR 12/12/2021 Criteria >0. 9 <0. 05 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 101

Confirmatory Factor Analysis n Example: Test Scores (CFA) M P C E H F Covariance Matrix (n=200) M P C E H F 4. 00 2. 48 2. 16 1. 28 1. 136 1. 48 4. 00 2. 04 1. 52 1. 404 1. 72 4. 00 1. 44 1. 334 1. 62 4. 00 2. 744 2. 92 4. 00 2. 94 4. 00 M: mathematics, P: physics, C: chemistry E: English, H: history, F: French; source: Sharma (1996) 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 102

Confirmatory Factor Analysis n Example: Test Scores (CFA) n n Model: one-factor model with 6 indicators P=6 and m=1 covariance matrices of construct and error are assumed to be symmetric The scale of the latent construct is assumed be the same as the first indicator variable (score of mathematics) n 12/12/2021 X 1 = F + e 1 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 103

Confirmatory Factor Analysis n Example: Test Scores (CFA) n Standardized factor loadings n n Estimates of parameters are standardized such that the variance of the construct is 1 Completely standardized factor loadings n n 12/12/2021 Estimated factor loadings are standardized such that both variances of the construct and indicators are 1. Estimated factor loadings after complete standardization should be between -1 and 1 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 104

Confirmatory Factor Analysis n Example: Test Scores (CFA) Estimates of factor loading (MLE) M P C E H F Completely Unstandardized Standardized 1. 000 0. 914 0. 457 1. 134 1. 037 0. 518 1. 073 0. 981 0. 491 1. 786 1. 633 0. 816 1. 770 1. 618 0. 809 1. 937 1. 771 0. 886 n 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D t-statistics 5. 21 5. 046 6. 393 6. 375 6. 533 105

Confirmatory Factor Analysis n Estimated error covariance matrix M P C E H M 0. 791 0 0 P 0. 731 0 0 0 C 0. 759 0 0 E 0. 333 0 H 0. 345 F The variance of the construct has been standardized unit variance 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D F 0 0 0. 216 to have 106

Confirmatory Factor Analysis n Estimates: n n Estimated factor loading are between -1 and 1 No negative estimates of variances of the construct and error terms Variances of the error terms are all smaller than 1 Estimates of factoring loadings and covariance matrices of the construct and error terms are admissible 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 107

Confirmatory Factor Analysis n Goodness of Model Fit Measures 2 test GFI EGFI RGFI AGFI EAGFI RAGFI 12/12/2021 value 113. 02 (9 df) with a p-value < 0. 001 0. 822 0. 985 0. 835(0. 822/0. 985) 0. 584 0. 965 0. 605 Recommendation reject the model < 0. 9 poor fit < 0. 8 poor fit Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 108

Confirmatory Factor Analysis Goodness of Model Fit n Measures RMSR (Multiple correlation)2 M P C E H F Coeff of Determination 12/12/2021 value Recommendation 0. 507 Poor fit 0. 209 0. 269 0. 241 0. 667 0. 655 0. 784 0. 895 <0. 5 >0. 8 no good for construct OK for construct as whole Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 109

Confirmatory Factor Analysis n Goodness of Model Fit Measures NCPh MDN NCPn TLI RNI 12/12/2021 value 0. 520 0. 771 2. 748 0. 685 0. 811 Recommendation >0. 10 poor fit <0. 90 poor fit Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 110

Confirmatory Factor Analysis n n Degrees of freedom under one-factor model n Number of free parameters: 5 factor loadings, 1 variance of construct and 6 variances of error terms = 5+1+6 =12 n Dfh = 6(6+1) -12 =9 Degrees of freedom under null model n Number of free parameters: 6 variances of error terms = 6 n Dfn = 6(6+1) -6 =15 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 111

Confirmatory Factor Analysis n n n 2 n = 564. 67 and 2 h = 113. 02 NCPn = (564. 67 -15)/200 = 2. 748 NCPh = (113. 02 -9)/200 = 0. 520 MDN = exp[-0. 520/2] = 0. 771 TLI = [2. 748/15 – 0. 520/9]/[2. 748/15)] = 0. 685 RNI = [2. 748 – 0. 520]/2. 748 = 0. 811 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 112

Confirmatory Factor Analysis n Example: Test Scores (CFA) n Model: Two-factor model with 6 indicators and correlated construct. P=6 and m=2 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 113

Confirmatory Factor Analysis n Example: Test Scores (CFA) n n n Covariance matrices of construct and error are assumed to be symmetric Correlation between two constructs The scale of the latent constructs is assumed be the same as the first and 4 th indicator variable (score of mathematics and English) 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 114

Confirmatory Factor Analysis n n Example: Test Scores (CFA) Estimates of factor loading (MLE) M P C E H F 12/12/2021 Completely Standardized (F 1) 0. 776 0. 785 0. 684 0 0 0 Completely Standardized (F 2) 0 0. 823 0. 822 0. 894 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D t-statistics 0 9. 210 8. 610 0 12. 933 13. 939 115

Confirmatory Factor Analysis n Estimated error covariance matrix M 0. 398 P 0 0. 384 C 0 0 0. 533 E 0 0. 323 H 0 0 0. 324 M P C E H F The variance of the construct has been standardized unit variance 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D F 0 0 0. 200 to have 116

Confirmatory Factor Analysis n Estimated covariance matrix of two construct Quantitative Verbal Quantitative 1 Verbal 0. 568 1 The variance of the construct has been standardized to have unit variance 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 117

Confirmatory Factor Analysis n Goodness of Model Fit Measures 2 test value 6. 05 (8 df) GFI EGFI RGFI AGFI EAGFI RAGFI 0. 990 1. 000(0. 990/0. 990) 0. 974 0. 965 1. 010 12/12/2021 Recommendation with a p-value =0. 642 Fail to reject the model > 0. 9 good fit > 0. 8 poor fit Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 118

Confirmatory Factor Analysis Goodness of Model Fit n Measures RMSR (Multiple correlation)2 M P C E H F Coeff of Determination 12/12/2021 value Recommendation 0. 111 Fair fit 0. 602 0. 616 0. 457 0. 667 0. 676 0. 800 0. 972 >0. 5 <0. 5 >0. 8 OK for construct no good for construct OK for construct as whole Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 119

Confirmatory Factor Analysis n Goodness of Model Fit Measures NCPh MDN NCPn TLI RNI 12/12/2021 value 0. 000 1. 000 2. 748 1. 000 Recommendation <0. 10 good fit >0. 90 good fit <0. 90 poor fit >0. 90 good fit Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 120

Confirmatory Factor Analysis n n If the null hypothesis is true, the expected value of NCP is 0 Due to sampling errors, it is possible to obtain negative estimates of NCP is set to be 0 and MDN, TLI and RNI will be 1 This represents an almost perfect model fit 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 121

Confirmatory Factor Analysis Reliability of a given construct (Sum of factor loadings)2/[(Sum of factor loadings)2+sum of error n variance] Quantitative = [0. 776+0. 785+0. 684]2/ [(0. 776+0. 785+0. 684)2+(0. 398+0. 384+0. 553)] =0. 793 Verbal = [0. 823+0. 822+0. 894]2/ [(0. 823+0. 822+0. 894)2+(0. 323+0. 324+0. 200)] =0. 884 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 122

Summary n n n Difference between principal component analysis and factor analysis Exploratory factor analysis Confirmatory factor analysis Linear structural relations Examples 12/12/2021 Copyright by Jen-pei Liu, Ph. D and Wei-Chu Chie, MD, Ph. D 123