HMS 320 Experimental Preexperimental and Quasiexperimental designs Internal

HMS 320 ØExperimental, Pre-experimental, and Quasi-experimental designs ØInternal Validity ØExternal Validity

Experimental • Treatment group • Control group • Random selection and assignment • Differences in groups caused by – Treatment (independent variable) – Error (random, presumably) • Reduce the error to see the treatment effect

Errors • Errors that occur because of selection bias • Errors that occur because of un-controlled variables – Known as threats to internal validity – Alternative explanations – Extraneous variables

Threats to Internal Validity • Alternative explanations • Alternative hypotheses • The DV is NOT dependent upon the IV but instead is dependent on something else

Subject characteristics Examples: • • • Age Gender Grade in school Intelligence SES

Loss of subjects Mortality (death) Drop out of school Move out of town Stop participating

Location • Environmental factors – Factors related to the setting • Lighting • Noise • Changes in setting

Instrumentation • Instrument decay – The instrument changes over time • grading papers, fatigue of teacher • Battery grows weaker – Data collector characteristics • Gender • Authority figure – Data collector bias • Different administrators of test • Inadvertent body language • Training can overcome this

Testing effect • Being exposed to the testing has an effect – Pre-test informs participants – Test-retest, first exposure provides practice

History • Events that occur to the participants during the span of the data collection • In experimental study, one group has a different experience than another group For example, if a researcher is evaluating a drug prevention program, and during the time of the study a popular celebrity dies from a drug overdose, the result of the study may be due to the reaction to the death rather than the intervention.

Maturation • The participants change over time – over the span of the data collection – Institute a character education program for 8 th graders. Over the year’s time, the 8 th graders experience perceive themselves as more able to stand up for themselves

Attitude of Participants • Hawthorne Effect • In experimental study, one group finds out the nature of the other group

Regression to the Mean • The Pre-test differs from the post-test, score moves to the overall average • No where to go but up (down).

Implementation • Usually shows up with experimental groups • Researcher may favor one method over another and subtly sets up one group for better performance • The difference between the selection of two groups is not random but an unrelated factor

Name that Threat 1. The pre test allows or cues the participants into learning the nature of the study.

Name that Threat 2. Two existing groups are compared with respect to their scores on an achievement test.

Name that Threat 3. A car accident occurs outside the school during an algebra test for one class, but another class completes it un-interrupted.

Name that Threat 4. An art program is instituted at two grade levels, 2 nd grade and 7 th grade. The 7 th graders show a much greater change in their skills than the 2 nd graders. Conclusion is that the program is more appropriate for 7 th graders.

Name that Threat 5. High school students in private schools score better on an instrument designed to test for financial awareness than students in a public high school.

Name that Threat 6. The scorers of an exam unconsciously grade the exam papers for a comparison group in a way the some students receive higher grades than another group of students.

Threats to External Validity • External validity = generalizing from the sample to the population • Threats include: – Selection bias – Reactive effects of experimental arrangements – Reactive effect of testing (pretest sensitization) – Multiple-treatment interference

Pre-experimental • “Pre” in the sense of “primitive” designs with no experimental features • One group • Case study • Static-group comparison Useful but can’t determine cause-effect Used in pilot studies

Quasi-experimental • Non-random selection and/or non-random assignment to groups – Non-equivalent groups – Equivalent-time samples design Baseline – know the term