Summary of Lecture 7 Part I True Score

Summary of Lecture 7: Part I True Score Variance & Reliability:

Summary of Lecture 7: Part II Example: Reliability of a single test with a single latent construct:

Summary of Lecture 7: Part III Reliability of a single test with multiple latent constructs: Matrix computations The term: represents the true score variance:

Summary of Lecture 7: Part IV Reliability of a single test with multiple latent constructs: Matrix computations We thus have: q True score variance: q Variance of Y: q Reliability of Y:

Summary of Lecture 7: Part V General comments concerning reliability: 1. The concept of reliability refers to the test and not to the construct, i. e. , a test can be (un-) reliable but not a construct. 2. Reliability is a population dependent concept: It can be estimated only if construct values vary (within a population). Otherwise the reliability is 0. 3. Reliability is a theoretical construct that cannot be observed directly. It can be measured only in a model dependent way, i. e. , assumptions about the measurement model are required and the measured reliability depends on the correctness of these assumptions.

Summary of Lecture 7: Part VI q Traditional approaches to measure reliability: q q q Test-Retest Method: Same test applied twice. Parallel forms: Two equivalent versions of the test. Test halves: Test is split in 2 halves (e. g. even vs. uneven). q The traditional approaches are based on the assumption that test are parallel. q Traditional approaches conceal the fact that reliability can be measured only in a model dependent way.

Summary of Lecture 7: Part VII o

Summary of Lecture 7: Part VIII o

Summary of Lecture 7: Part IX Limitations of traditional measures of the reliability of sum scores: 1. The traditional measures of test scores (Spearman. Brown, coefficient and Guttman’s 2 are unbiased only in case of assumptions to be met: q parallel tests in case of Spearman-Brown. q -equivalent tests in case of and 2. 2. For congeneric measures or measures loading on more than one latent construct, and without correlated errors and 2 underestimate the true reliability. 3. 2 underestimates the true reliability less than .

Summary of Lecture 7: Part X Limitations of traditional measures of the reliability of sum scores: 4. In case of correlated errors the three traditional estimates of the reliability of sums can overestimate the reliability of the sum (An example is given below and in the exercises). 5. The traditional measures use observed correlations and (co-) variances and not the model implied correlations and (co-) variances. The latter are to be preferred, in case of the model being approximately correct, since they are more robust. 6. If an analysis of the covariance structure of the tests has been performed an unbiased estimate can be computed.