Randomized Experimental Design What is an Experiment Campbell

Randomized Experimental Design

What is an Experiment? • Campbell & Stanley stressed random assignment to experimental treatments. • I stress manipulation of the independent variable. • Quasi-Experiments: C&S’s term for research where – there is a manipulated IV – but not random assignment to groups

Random Selection • Refers to the selection of data points from a population into a random sample. • This selection procedure will be random if each possible sample of size N is equally likely to be sampled. • Random samples should be representative of the population. • Our inferential statistics assume random sampling.

Note. The tabled values are probabilities. • Y is random, X is not. • In Y, each time a single score is sampled, all scores in the population are equally likely to be sampled.

Random Assignment • Refers to the assignment of subjects to treatment conditions. • Allows us to consider the populations (subjects who will get special treatment and those who will not) as equivalent prior to treatment. • The samples will likely differ a little.

Two Basic Randomized Designs • Randomized Pretest-Posttest Control Group Design R O X O R O O • Randomized Posttest Only Control Group Design R X O R O

Noise Reducing Designs • These designs reduce noise (error variance) and thus increase power.

Randomized Blocks Designs • Matched pairs, randomized blocks. • Repeated measures or within-subjects. • Variance due to the blocking variable is removed from error variance. Blocks Error Treatment Blocks Treatment

Analysis of Covariance • Change a noise-producing extraneous variable into a covariate that is included in the statistical model. • Must be able to measure the covariate. • Variance due to covariate is removed from the error variance. • Can have more than one covariate.

Factorial ANOVA • Convert a categorical extraneous variable to an ANOVA factor. • Variance due to that factor will be removed from the error term. 2 x 2 Factorial Design R X 11 O R X 12 O R X 21 O R X 22 O

Other Randomized Designs Solomon Four Group Design • Controls threats to internal and external validity as well as the posttest only control group design. • But has greater power. • And greater cost – Need more data – More complex analysis R O X O R O O R X O R O

Solomon Four Group Design ANOVA • Arrange all four groups’ posttest scores into a 2 x 2 ANOVA. • Pretested or Not x Experimental Treatment or Not. • Significant Interaction – Testing x Treatment threat to External Validity • Main effect of pretesting. • Main effect of treatment

Treatment effect but no testing or Testing x Treatment interaction Pretest means in parentheses

Treatment and testing effects but no Testing x Treatment interaction

Treatment and testing effects and a Testing x Treatment interaction

Solomon Four Group Design Pretest-Posttest Analysis • To gain power, analyze the pretestposttest portion of the design with – ANCOV, using pretest scores as covariate – Mixed factorial ANOVA – planned comparisons using t • control versus treatment change scores (independent t) • pre versus post for control group (correlated t) • pre versus post for treatment group (correlated t)

Randomized Switching. Replications Design R O X O O R O O X O • Attempt to control social threats to internal validity. • Both groups get the special effect, one early, one later. • May still be social effects with respect to who gets it first.

Switching Replications Temporary Treatment Effect • Group 1 got the treatment first. • Treatment is anxiety-reducing drug • DV = measure of anxiety reported by patient

Switching Replications Persistent Treatment Effect • Treatment is psychotherapy • DV = measure of anxiety reported by patient • A third posttest could show the effect does not last indefinitely.

Switching Replications Continuing Treatment Effect • Treatment = cognitive psychotherapy • Anxiety continues to decline beyond the first post -treatment observation, as patients get better at employing the cognitive technique.