Applied Psychometric Strategies Lab Applied Quantitative and Psychometric
Applied Psychometric Strategies Lab Applied Quantitative and Psychometric Series Zijia Li, Ph. D Caroline J. Gooden, Ph. D Michael D. Toland, Ph. D Measurement Invariance with Categorical Indicators December 6, 2016
Today’s Agenda • What is measurement invariance (MI) with categorical items? • Why is testing for MI important? • What are the levels of MI? • How is MI tested? • Provide an applied example 2
What Is MI? A technique that helps determine if the same unobserved variable is being measured across multiple groups (or time) within a confirmatory factor analysis (CFA) framework 3
Why is Testing for MI Important? When MI is evidenced, it assures that • Group comparisons are meaningful • We are measuring the same trait across groups • Group differences reflect true group differences 4
What If There Is No Evidence for MI? If MI assumptions aren’t tenable, then • Scores from multiple groups don’t represent construct equally well • Observed group differences cannot be assumed to be accurate • Findings may be unsubstantiated or invalid 5
Why Don’t Some Studies Test for MI? • Lack of sample size for each group being compared to ensure results are trustworthy • Lack of knowledge about how to test for MI 6
What are Some Examples Where MI Is Tested? Compare different groups across the same instrument • • Gender Race/Ethnicity Age Treatment condition Language Degree level Other 7
What are the Levels of MI? Grimm et al. (2017) 1. Dimensional invariance 2. Configural invariance 3. Weak factorial invariance 4. Strong invariance 5. Strict factorial invariance NOTE: Each level assumes evidence for MI at the previous level 8
1. Dimensional Invariance • Typically tested with exploratory factor analysis (EFA) Ø Same number of latent factor(s) Ø Same item-factor structure Ø Factor loadings equal across groups Ø Item thresholds equal across groups Ø Item residuals equal across groups 9
Conceptual Representation of Dimensional Invariance 10
Levels 2 -5 of MI Tested within a CFA framework: 2. Configural invariance 3. Weak factorial invariance 4. Strong invariance 5. Strict factorial invariance 11
Levels 2 -5 are tested via a CFA Framework Degree of Invariance Item Factor Loading ( ) Item Threshold ( ) Item Residuals (u) Configural Invariance Free Weak Factorial Invariance Equal Free Strong Invariance Equal Free Strict Factorial Invariance Equal 12
2. Configural Invariance Ø Same number of latent factor(s) across groups Ø Same item-factor structure across groups Ø Factor loadings equal across groups Ø Item thresholds equal across groups Ø Item residuals equal across groups 13
Conceptual Representation of Configural Invariance Group 2 Group 1 ϕ 1 η 1 λ 41 λ 11 λ 31 τ11 τ21 ϕ 2 η 2 1 1 λ 42 τ31 λ 12 τ41 λ 22 τ12 λ 32 τ22 τ42 τ32 y 11 y 21 y 31 y 41 y 12 y 22 y 32 y 42 u 11 u 21 u 31 u 41 u 12 u 22 u 32 u 42 14
3. Weak Factorial Invariance Ø Same number of latent factor(s) Ø Same item-factor structure Ø Factor loadings equal across groups Ø Item thresholds equal across groups Ø Item residuals equal across groups 15
Conceptual Representation of Weak Factorial Invariance Group 2 Group 1 ϕ 1 η 1 λ 41 λ 11 λ 31 τ11 τ21 ϕ 2 η 2 1 1 λ 42 τ31 τ41 λ 21 λ 12 λ 22 τ12 λ 32 τ22 τ42 τ32 y 11 y 21 y 31 y 41 y 12 y 22 y 32 y 42 u 11 u 21 u 31 u 41 u 12 u 22 u 32 u 42 16
4. Strong Invariance Ø Same number of latent factor(s) Ø Same item-factor structure Ø Factor loadings equal across groups Ø Item thresholds equal across groups Ø Item residuals equal across groups 17
Conceptual Representation of Strong Invariance Group 2 Group 1 ϕ 1 η 1 λ 41 λ 11 λ 31 τ11 τ21 ϕ 2 η 2 1 λ 42 τ31 τ41 λ 21 λ 12 λ 22 λ 32 1 τ12 τ22 τ32 τ42 y 11 y 21 y 31 y 41 y 12 y 22 y 32 y 42 u 11 u 21 u 31 u 41 u 12 u 22 u 32 u 42 18
5. Strict Factorial Invariance Ø Same number of latent factor(s) Ø Same item-factor structure Ø Factor loadings equal across groups Ø Item thresholds equal across groups Ø Item residuals equal across groups 19
Conceptual Representation of Strong Invariance Group 2 Group 1 ϕ 1 η 1 λ 41 λ 11 λ 31 τ11 τ21 ϕ 2 η 2 1 λ 42 τ31 τ41 λ 21 λ 12 λ 22 λ 32 1 τ12 τ22 τ32 τ42 y 11 y 21 y 31 y 41 y 12 y 22 y 32 y 42 u 11 u 21 u 31 u 41 u 12 u 22 u 32 u 42 20
Implications Degree of Invariance Recommended Implications Configural Invariance Separate group analyses; no group comparison or full group Weak Factorial Invariance Variances and covariances can be compared at the latent level Strong Invariance Strict Factorial Invariance analyses at observed or latent level are recommended (via SEM) Means, variances, and covariances can be compared at the latent level (via SEM) True group differences only source of difference in means, thus observed and latent comparison can be made for variances, covariances, and means 21
Applied Example in Mplus • 976 girls vs. 1, 153 boys • Child assessment measure of language – 1 language factor – 4 items • 4 Likert categories per item 22
Model comparison • Chi-square DIFFTEST (ɑ =. 05) • Modification Indices (Liu, Millsap, West, Tein, Tanaka, & Grimm, 2016) • ΔCFI ≤ -. 002 (Meade et al. , 2008) • ΔTLI = 0 (Marsh, Lüdtke, Muthén, Asparouhov, Morin, Trautwein, & Nagengast, 2010) • ΔRMSEA ≥. 007 (Meade et al. , 2008) 23
Mplus Syntax 24
Mplus Syntax: Configural Model 25
Mplus Output: Model Fit Information for Configural Model 26
Mplus Syntax: Weak Factorial Invariance Model 27
Mplus Output: Model Fit Information for Weak Factorial Invariance Model 28
Mplus Output: Modification Indices for Weak Factorial Invariance Model (By statement) 29
Mplus Syntax: Strong Invariance Model 30
Mplus Output: Model Fit Information for Strong Invariance Model 31
Mplus Output: Modification Indices for Strong Invariance Model ($ statement) 32
Mplus Syntax: Strict Factorial Invariance Model (1 of 2) 33
Mplus Syntax: Strict Factorial Invariance Model (2 of 2) 34
Mplus Output: Model Fit Information for Strict Factorial Invariance Model (2 of 2) 35
Mplus Output: Modification Indices for Strong Invariance Model (Residual terms) 36
Conclusions • Evidence for strict factorial invariance for language • Gender difference- only source of difference found in the mean, variance, and covariance for language 37
Helpful Resources • • Buhs, E. S. , Mc. Ginley, M. & Toland, M. D. (2010). Overt and relational victimization in Latinos and European Americans: Measurement equivalence across ethnicity, gender, and grade level in early adolescent groups. The Journal of Early Adolescence, 30, 171 -197. doi: 10. 1177/0272431609350923 Chen, F. F. , & West, S. G. (2008). Measuring individualism and collectivism: The importance of considering differential components, reference groups, and measurement invariance. Journal of Research in Personality, 42, 259 -294. Cheung, G. W. , & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233 -255. Marsh, H. W. , & Grayson, D. (1995). Latent variable models of multitrait–multimethod data. In R. H. Hoyle (Ed. ), Structural equation modeling: Concepts, issues and application (pp. 177– 198). Thousand Oaks, CA: Sage. • Grimm, K. J. , Ram, N. , & Estabrook R. (2017). Growth modeling: Structural equation and multilevel modeling approaches. New York, NY: The Guilford Press. • • Millsap, R. E. (2011), Statistical approaches to measurement invariance. NY, NY: Routledge. Millsap, R. E. (2010). Testing measurement invariance using item response theory in longitudinal data: An introduction. Child Development Perspectives, 4: 5– 9. doi: 10. 1111/j. 1750 -8606. 2009. 00109. x Sessions, J. , Finney, S. J. , & Kopp, J. P. (2016). Does the measurement or magnitude of academic entitlement change over time? Measurement and Evaluation in Counseling and Development, 1 -15. Toland, M. D. , & Kupzyk, K. (2007, February). Approaches for evaluating measurement invariance. Presentation for the Nebraska Center for Research on Children, Youth, Families, and Schools Research Methodology Series, Lincoln, NE. van de Schoot, R. , Lugtig, P. , & Hox, J. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9, 486 -492. doi: 10. 1080/17405629. 2012. 686740 Vandenberg, R. J. , & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4 -70. doi: 10. 1177/109442810031002 38 Widaman, K. F. , & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. The Science of Prevention, 281 -324. • • •
39
- Slides: 39