Chapter 3 Practice of Psychometrics Components of a

Chapter 3: Practice of Psychometrics Components of a psychometric analysis: 1. Construction of a measurement model that represents the important factors exerting an influence on the measure, and estimating the model parameters by fitting the model to data. 2. Test of the measurement model. 3. Evaluation of the characteristics of the test items. 4. Prediction of latent construct scores. 5. Comparison of different groups.

Chapter 3: Practice of Psychometrics q Construction of the measurement model and parameter estimation: Ø Construction on the basis of theoretical considerations. Ø Parameter estimation is performed by means of an adequate program (lavaan, ltm). Ø Parameters contain information about test items and the distribution of latent constructs and relationship between latent constructs and test items. q Test of the model: Ø Assessment regarding the content of the model. Ø Evaluation of how well the model explains the data. Ø Consideration of model variants.

Chapter 3: Practice of Psychometrics q Evaluation of item characteristics: Ø Reliability, validity (classical test theory). Ø Information function (probabilistic test theory). Ø Range of latent scores where items function optimal (probabilistic test theory). q Prediction of latent scores for single test takers: Ø On the basis of test results and the estimated parameters latent scores are predicted. q Comparison of latent groups: Ø Fairness of tests. Ø Do item function differently in different groups (DIF = different item functioning), Measurement invariance.

Chapter 3: Structure of psychometric models The general measurement model: Variables q Latent abilities (eta) Ø Multivariate normal distributed random variables. q Hidden responses (xi) Ø Represent hidden continu ous response processes. q Error variables (epsilon) Ø Represent unmeasured influences. q Observed variables Y

Chapter 3: Structure of psychometric models The general measurement model: Three components: 1. Latent constructs and their relationships. 2. Observed measures. 3. Response model: Mapping latent vari ables to observed measures.

Chapter 3: Structure of psychometric models The general measurement model: q Circles: Latent variables q Rectangles: Observed variables q Triangles: Fixed values and not random variables Ø Either constants (e. g. 1) or parameters. q Greek letters outside circles: Parameters Ø Fixed parameters are represented by numbers

Chapter 3: Structure of psychometric models The general measurement model: q Covariance arcs Ø Unmodeled covariances between variables Ø Exist only between latent abilities and, possibly, between errors. q Arrows Ø Represent linear causal influences (details later). q Red sigmoid curves Ø Represent response functions ICC = Item characteristic curves.

Chapter 3: Structure of psychometric models The general measurement model: Parameters q Variance of and covariances between latent abilities: (phi) q Mean parameters of latent abilities (mu) q Variances of and covari ances between error variables: (theta) q Regression weights: Ø Loading coefficients: (lambda) Ø Item difficulties: (beta) q Discrimination parameters of response functions (alpha)

Chapter 3: Structure of psychometric models The general measurement model: Linear equations q The hidden responses are linear combinations of: Ø Latent abilities Ø Errors Ø Items difficulties q The linear equations: (more explanations later)

Chapter 3: Structure of psychometric models The general measurement model: Response functions: q Logistic function: q Normal ogive: q No response function: CTT

Chapter 3: Structure of psychometric models The general measurement model: Response functions: The problem of indeterminacy q Response function: The participant selects response category c 1 if the hidden re sponse j of that person j surpasses a certain threshold j, other wise she selectsresponse category c 2. q The problem of indeterminacy concerning the source of randomness: Ø Random thresholds Ø Random error associated with hidden responses

Chapter 3: Structure of psychometric models The general measurement model: Response functions: The problem of indeterminacy q Random thresholds: q Random error in the hidden response:

Chapter 3: Structure of psychometric models The general measurement model: Response functions: The problem of indeterminacy q Random error in the hidden response: q Conclusion: Location of the random component cannot be determined uniquely.