Internal Validity Questions Did the treatment cause a
- Slides: 24
Internal Validity Questions Did the treatment cause a change in behavior? Treatment Causes Change in Behavior
8 Threats to Internal Validity (Cambell & Stanley, 1963) o o o o History Maturation Testing Instrumentation Regression Selection by maturation interactions Mortality
Identifying the Spurious 8 o Individuals changing without treatment 1. 2. 3. o Maturation History Testing Measurement error disguised as treatment effect 4. 5. Instrumentation Regression
Identifying the Spurious 8 (cont) o Non-treatment differences between groups 6. 7. 8. Mortality Selection by maturation interactions
Ruling out the Spurious 8 o o Find out what threats are automatically ruled out by your design Use logic to try to rule out the remaining threats
External Validity Questions: Can the Results be Generalized? o o Can the results be generalized to other participants? Can the results be generalized to other settings? Results from the research study (a “Mini-world”) Generalize Larger World
Threats to Internal Validity History o o o A threat when an observed effect might be due to an event which takes place between the pre-test and post-test; when the event is not of research interest at that time In the lab we control for this by “insulating” the P Rarely able to fully control for this outside lab
Maturation o o o A threat when an observed effect might be due to the respondent’s changing with the passage of time per se. Not specific to events Examples n n n Aging (wiser, stronger) More experience on DV Rested, tired Time of the month Etc. etc.
Testing o o o A threat when observed effect may be due to the number of times particular responses are measured. Frequently familiarity with a test or scale could affect performance or results because past responses were remembered Error responses might be remembered (very likely)
Instrumentation o A threat when an effect might be due to a change in the measuring instrument between pre-test and post-test and not to the IV(s) at each time interval
Statistical Regression o o o A threat when an effect might be due to respondent’s being classified into experimental groups, highs, or lows, based on pretest position in the distribution High on pretest, more like to score lower on posttest Low on posttest, more likely to score higher on posttest
Selection o o o A threat when an effect may be due to the difference between kinds of people in one experimental group as opposed to another group Pervasive problem in quasi-experimental designs, P’s are not randomly assigned to groups Serious potential challenge if not dealt with
Interactions with selections o Many threats to internal validity can interact with selection to produce forces that might spuriously appear as treatment effects n Selection-maturation Selection-history n Selection-instrumentation n
The King of Internal Validity: Randomization o If you can randomize, then there are no threats due to: n n n o Selection Maturation Selection-maturation interaction If P’s all experience same testing conditions, then there is also no threat from: n n Testing Instrumentation
Threats to External Validity o Reactive Testing n o Pretest may cause increase or decrease in P’s sensitivity to the IV Interaction of Setting and Treatment n n Results would only generalize to other Ps in very similar settings For example: other colleges, hospitals, mental health centers, etc.
Coding X=IV O=Measurement of DV R=Randomization
Pre-Experimental Designs One Shot Case Study ----X—O • Nothing is controlled • Why do it?
One-Group Pretest-Posttest Study: o O---X---O What is Controlled? o Static-Group Comparison • Selection X---O -----O • Mortality What is Controlled? • Reactive arrangement? ? ? • History • Nothing else is controlled • Maturation • Testing • Instrumentation • Nothing else is controlled
True Experimental Design Pre-test—Post-test Control Group Design R R O O X --- O O What is Controlled? • All threats to internal validity • Does not control threats to external validity
Solomon Four-Group Design R R O O ----- X --- O O What is Controlled? • All threats to internal validity • Threat of interaction of testing and X • Might control for interactions of selection and X • Might control for Multiple interactions
Post-test Only Control Group Design R R ----- X --- O O What is Controlled? • All threats to internal validity • Interaction of testing and X • Most common design in experimental design
Quasi-Experimental Designs Time Series Design OOOOOXOOOOO What is Controlled? • Maturation • Testing • Regression • Selection • Mortality • Interaction with selection, maturation, etc.
What is not covered? n n History Questionable on Instrumentation Interaction of testing and treatment Questionable on selection X interaction and multiple interactions
Non-Equivalent Control Group Design O O X --- What is Covered? • History maturation • Testing • Instrumentation • Selection • Mortality O O What is Not Covered? • Interaction of Testing and treatment • Interaction of Selection, maturation, etc. • Questionable for regression, reactive arrangements
- Criterion validity vs predictive validity
- Selection by maturation interaction
- Internal validity
- Internal validity definition
- What is external validity
- Ter thin client
- Greylag goose egg-retrieval behavior
- Proximate cause and ultimate cause
- Kinesis and taxis
- Internal control structure
- Importance of vouching
- Dbq treaty of versailles
- How did the triple entente cause ww1
- How could the french revolution been avoided
- War of 1812 free seas and trade
- Foto de tsunami
- How did franz's feelings about mr.hamel and school change
- Hình ảnh bộ gõ cơ thể búng tay
- Bổ thể
- Tỉ lệ cơ thể trẻ em
- Chó sói
- Glasgow thang điểm
- Chúa yêu trần thế alleluia
- Môn thể thao bắt đầu bằng chữ đua