Modeling item response profiles using factor models latent

Modeling item response profiles using factor models, latent class models, and latent variable hybrids Dena Pastor James Madison University pastorda@jmu. edu

Purposes of the Presentation • To present the model-implied item response profiles (IRPs) that correspond to latent variable models used with dichotomous item response data • To provide an example of how these models can be used in practice

Proportion Responding Correctly Item Response Profiles (IRPs) 1 0, 9 0, 8 0, 7 0, 6 0, 5 0, 4 0, 3 0, 2 0, 1 0 1 2 Item Number 3 4

Pattern Differences IRPs for classes of examinees with different patterns

Elevation Differences IRPs for classes of examinees with the same pattern, but differences in elevation

Latent Variable Model C NON-PARALLEL C is a nominal latent variable PARALLEL Latent Class Model C is a latent categorical variable with as many levels as # of classes C is a ordinal latent variable

Exploratory Process • In latent class modeling a variety of models are fit to the data with differing numbers of classes – 1 -class model, 2 -class model, 3 -class model, etc. • Use fit indices and a priori expectations to determine the number of classes to retain • Can allow latent categorical variable to be nominal and examine resulting profiles; can also constrain latent categorical variable to be ordinal

Alternative Model for Parallel Profiles Do we have 3 classes, with no variability within class? OR Factor Model F is a latent continuous variable F Do we have 1 profile with systematic variability within class?

Different Models for Different IRPs …+ within profile variability 1 profile… LCM: 1 class 2 parallel profiles… Factor Model …+ within profile variability Semi-parametric Factor Model LCM: 2 classes (C is ordinal) 2 non-parallel profiles… LCM: 2 classes (C is nominal) …+ within profile variability Factor Mixture Model Latent Variable Hybrids

Number of profiles? Decisions 1 Models (number of classes) Nature of profile differences? Parallel Systematic variability within profiles? no IRPs 1+ Latent class model (LCM) Systematic variability within profiles? yes Factor model (FM) no LCM with parallel profiles Nonparallel yes Semiparametric factor model (SPFM) no LCM with nonparallel profiles yes Factor mixture model (FMM)

Latent class model (LCM) C 2 classes F 1 class: Factor Model! F C 2 classes, w/in class factor variance = 0 C F F C Factor mixture model (FMM) Semi-parametric factor model (SPFM) C 2 classes 1 class: Factor Model! F C 2 classes, w/in class factor variance = 0 C F

Marginal probability of getting an item correct is sum across classes of probability of getting item correct conditional on class membership Conditional probability differs across models Factor mixture model (FMM) Semiparametric factor model (SPFM) Latent class model (LCM) C F C

Latent class model (LCM) IRP Latent Variable Distribution Path diagram C C is ordinal C is nominal

Semi-Parametric Factor Model (SPFM) IRP Latent Variable Distribution Measurement Invariance Same measurement model parameters (thresholds, loadings) for each class Path diagram C F Quantitative differences between classes

Factor Mixture Model (FMM) IRP Latent Variable Distribution Path diagram C F Measurement Non-Invariance Different measurement model parameters (thresholds, loadings) for each class Qualitative differences between classes

Example • 9 dichotomously scored items measuring 3 aspects of psychosocial research: 1. Confidentiality 2. Generalizability 3. Informed Consent • Sample 2, 259 incoming freshmen tested in low -stakes conditions prior to start of classes

Exploratory Model Selection • Exploratory model selection approach to answer the question, “What type and number of latent variables are most salient for our data? ” • Reasons to believe that IRPs would differ in pattern and/or elevation because students differ in: • Completion of psychosocial coursework • Effort they put forth on test

Model Fit Indices Model FM 1 f 2 f 3 f BIC SSA-BIC 25395 25338 25352 25270 25348 25243 LMR NA NA NA 9 19 29 39 49 25946 25445 25406 25377 25418 25918 25384 25314 25253 25262 NA 0. 00 0. 07 0. 01 0. 14 SPFM 1 f 2 c -12618 21 25399 25332 0. 01 FMM 1 f 2 c -12539 35 25348 25237 0. 00 LCM 1 c 2 c 3 c 4 c 5 c LL -12628 -12576 -12547 # paras 18 26 33 -12938 -12649 -12591 -12538 -12520

IRPs of 4 Class LCM 0. 18 0. 36 0. 25 0. 20 generalizability

2 -class FMM Factor Variability Within Each Clas 0. 44 0. 56 26. Which ethical practice is not considered by Marty? a) She failed to obtain informed consent from her participants b) She failed to randomly select participants c) … d) …

Visually Conveying Loading Information X Y

Validity Evidence for 2 -class FMM Solution X Y • Students with higher SAT-V scores, who reported put forth more effort on the test, and who have completed psychosocial coursework more likely to be in Class X • Positive relationship between SAT-V, coursework completion and factor scores in that class (negative relationship with effort) • Negative relationship between number of missing responses and factor scores in Class Y

Correspondence Between Models C&D from LCM, Y from FMM A&B from LCM, X from FMM X&Y from FMM with intervals

Parting Thoughts… • These models are like potato chips… – It was so much easier to settle on a brand of chip when I had a limited number of brands to choose from – But I also like having more brands because it increases my chances of finding the brand that is right for me – With all these brands, it is possible that some are selling essentially the same chip…. but which ones? – When two brands are essentially the same chip,

Questions? pastorda@jmu. edu Pastor, D. A. , & Gagné, P. (2013). Mean and covariance structure mixture models. In G. R. Hancock & R. O. Mueller (Eds. ), Structural Equation Modeling: A Second Course (2 nd Ed. ). Greenwich, CT: Information Age. Pastor, D. A. , Lau, A. R. , & Setzer, J. C. (2007, August). Modeling item response profiles using factor models, latent class models, and latent variable hybrids. Poster presented at the annual meeting of the American Psychological Association, San Francisco.