Ch 6 Reliability and Validity in Measurement and
Ch. 6: Reliability and Validity in Measurement and Research Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Validity and Reliability n n Validity: How well does the measure or design do what it purports to do? Reliability: How consistent or stable is the instrument? n Is the instrument dependable? Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Random and Systematic Error n Random Error: Chance fluctuations n n Tend to cancel out over repeated measurements. Systematic Error: Fluctuations that are slanted in a particular direction. n Also known as “bias” Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Types of Reliability n Test-Retest Reliability: Degree of temporal stability of the instrument. n n Assessed by having instrument completed by same people during two different time periods. Alternate-Forms Reliability: Degree of relatedness of different forms of test. n Used to minimize inflated reliability correlations due to familiarity with test items. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Types of Reliability (cont. ) n Internal-Consistency Reliability: Overall degree of relatedness of all test items or raters. n n Also called reliability of components. Item-to-Item Reliability: The reliability of any single item on average. n Judge-to-Judge Reliability: The reliability of any single judge on average. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Spearman-Brown Formula n n Used to estimate the internal consistency of a test or judges overall. The formula is: Where: SB = the overall internal consistency reliability n R n n = the total number of items in the test n rii = the average intercorrelation among the items or among the judges. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Reliability, Replication, and External Validity n n External Validity: The generalizability of an inferred causal relationship. The dependability of causal generalizations is based on replicable findings. n n But how know if have replicated a finding? Role of the effect size. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Validity in Test and Instrument Construction n n Content Validity: Is the relevant material adequately sampled? Criterion Validity: How well does the test correlated with outcome criteria? n n Concurrent Validity Predictive Validity Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Construct Validity in Test Development n Construct Validity: What does the test really assess? n n Does the test have the “ability to discriminate”? Establishing construct validity n n Convergent validity Discriminant validity Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
Validity and Causal Inference in Experimental Design n n External Validity Construct Validity: The validity of the hypothetical idea linking the IV and DV. Statistical Conclusion Validity: Are the statistical conclusions well-grounded? Internal Validity: Ability to rule out plausible rival hypotheses. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
- Slides: 10