1 Experiments and QuasiExperiments 2 Introduction Experiment using

2 Introduction • Experiment: using a controlled situation to observe a result • Involves

3 The Classical Experiment • Involves three major pairs of components: • Independent and

4 Variables, X and Y • X = Independent Variable (IV), cause, influencer •

5 Control and Experimental Groups • Experimental group – exposed to whatever treatment, policy,

6 Selecting Subjects • Decide on target population 1 st– the group to which

7 Hawthorne Effect • Pointed to the necessity of control groups • IV: improved

8 Placebo • We often don’t want people to know if they are receiving

9 Pre-Testing and Post-Testing • First, subjects measured on DV prior to association with

10 Double-Blind Experiment • Subjects and experimenters do not know who is in the

11 Experiments and Causal Inference • Experimental design ensures: • Cause precedes effect via

12 Internal Validity Threats (12) • Conclusions drawn from experimental results may not reflect

13 More Internal Validity Threats 4. Instrumentation – Changes in the measurement process 5.

14 Last, Internal Validity Threats 9. Diffusion/imitation of treatment – when E and C

15 Construct Validity Threats • Concerned with generalizing from experiment to actual causal processes

16 External Validity Threats • Significant for experiments conducted under carefully controlled conditions rather

17 Statistical Conclusion Validity Threats (Low Power) • Problem is likely when using small

18 Quasi-Experimental Designs • When? —randomization not possible • Quasi = “to a certain

Slides: 18

Download presentation

1 Experiments and Quasi-Experiments

2 Introduction • Experiment: using a controlled situation to observe a result • Involves taking and observing action • Great for hypothesis-testing • Theory-full

3 The Classical Experiment • Involves three major pairs of components: • Independent and dependent variables • Pre-Testing and Post-Testing • Experimental and Control groups • Randomization

4 Variables, X and Y • X = Independent Variable (IV), cause, influencer • Y = Dependent Variable (DV), effect, outcome

5 Control and Experimental Groups • Experimental group – exposed to whatever treatment, policy, initiative we are testing • Control group – very similar to experimental group, except that they are NOT exposed

6 Selecting Subjects • Decide on target population 1 st– the group to which the results of your experiment will apply • Cardinal rule – ensure that C and E groups are as similar as possible • Randomization helps towards this

7 Hawthorne Effect • Pointed to the necessity of control groups • IV: improved working conditions (better lighting) • DV: improvement in employee satisfaction and productivity • Workers were responding more to the attention than to the improved working conditions

8 Placebo • We often don’t want people to know if they are receiving treatment or not • We expose our control group to a “dummy” IV just so we are treating everyone the same • Medical research: participants don’t know what they are taking • Ensures that changes in DV actually result from IV and are not psychologically based

9 Pre-Testing and Post-Testing • First, subjects measured on DV prior to association with the IV (pre-tested) • Next, subjects are exposed to the IV • Third, subjects are remeasured in terms of the DV (post-tested) • Difference? --must be the intervention!

10 Double-Blind Experiment • Subjects and experimenters do not know who is in the control and experimental groups

11 Experiments and Causal Inference • Experimental design ensures: • Cause precedes effect via taking posttest • Empirical correlation exists via comparing pretest to posttest • No spurious 3 rd variable influencing correlation via posttest comparison between experimental and control groups, and via randomization

12 Internal Validity Threats (12) • Conclusions drawn from experimental results may not reflect what went on in experiment 1. History – external events may occur during the course of the experiment 2. Maturation – people grow 3. Testing – the process of testing and retesting

13 More Internal Validity Threats 4. Instrumentation – Changes in the measurement process 5. Statistical regression – Extreme scores regress to the mean 6. Selection bias – the way in which subjects are chosen 7. Experimental mortality – subjects may drop out prior to completion of experiment 8. Causal time order – ambiguity about order of stimulus and DV – which caused which?

14 Last, Internal Validity Threats 9. Diffusion/imitation of treatment – when E and C groups communicate, E group may pass on elements to C 10. Compensatory treatment – C group is deprived of something considered to be of value 11. Compensatory Rivalry – C group deprived of the stimulus may try to compensate by working harder 12. Demoralization – feelings of deprivation result in C group giving up

15 Construct Validity Threats • Concerned with generalizing from experiment to actual causal processes in the real world • Link construct and measures to theory • Clearly indicate what constructs are represented by what measures • Decide how much treatment is required to produce change in DV

16 External Validity Threats • Significant for experiments conducted under carefully controlled conditions rather than more natural conditions • But, this reduces internal validity threats! • A conundrum! • Suggestion – explanatory studies -> internal validity; applied studies -> external validity

17 Statistical Conclusion Validity Threats (Low Power) • Problem is likely when using small samples • With more cases, it is easier to see more differences

18 Quasi-Experimental Designs • When? —randomization not possible • Quasi = “to a certain degree” or, in short, “like”