Threats to External Validity External Validity Refers to

Threats to External Validity

External Validity Refers to the extent to which findings from a study can be generalized to situations or participants outside of the study.

Internal What is the cause-andeffect relationship? External Does this relationship hold in other situations?

Research Design in Perspective

Three Types of Generalizability Population: from our sample to the population Environmental or Ecological: from the lab to the “real world” Temporal: from the present to the future.

Four Threats to External Validity WAYS YOUR STUDY MAY DIFFER FROM ITS APPLICATION IN REAL LIFE.

1. Pretesting May actually change the subject in a way that we didn’t want. In the real world we would only manipulate the IV, not pretest. Example: Drug study. ◦ Participants come in for pre-testing of symptoms and to see a physician for baseline data. In the real world, patients don’t get all this extra attention, just the drug.

Solomon Four Group Design Allows us to see what would happen without pretesting. R = Random assignment O = Observation/measurement of DV X = Treatment/IV R O 1 R O 3 O 2 X R R O 4 O 5 X O 6

2. Interaction of Selection with Treatment When the treatment effect only occurs with a specific sample. Example: Accupuncture may only work with individuals who are highly accepting of non-western treatments.

3. Reactive Arrangements Conditions of the experimental setting that alter participant’s behavior. e. g. , Hawthorne Effect: Couples observed in a lab may respond well to marriage counseling techniques but the same techniques may not work as well if couples come in from the “real world/”

4. Multiple Treatment Interference A problem for repeated measures studies because in the real world people might only receive one treatment. e. g. , Marketing research may involve getting multiple opinions on multiple products but in the real world, people would only get the version that made it through market research.

Other Threats to Generalizability Using Convenience Samples -White Rats -College students -White -American

Best Defense Replicate with different samples, in different places and with systematic modifications to the design.

Programmatic Research A series of research experiments that deal with a related topic or question. Systematically rules out threats to internal and external validity.

In Defense of External Invalidity MOOK’S REASONS WHY WE DON’T ALWAYS NEED EXTERNAL VALIDITY

Reasons we might not care about generalizing v. We just want to know if something can happen even if it rarely does in the real world. v. We want to understand the specifics of a phenomenon under controlled conditions. v. We want to show that something can still happen even in an artificial setting or with a different population.

When do we need external validity? Only when we are trying to predict real-life behavior in the real world.

Will your group project have external validity? DOES IT MATTER?

Will your group project have internal validity? DOES IT MATTER?