Chapter 13 Matched Pairs WithinSubjects and Mixed Designs

Overview l l l The Matched Pairs Design Pure Within-Subjects Designs Randomized Within-Subjects Designs Counterbalanced Within-Subjects Designs Choosing a Design *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

The Matched Pairs Design l l l Procedure* Considerations in Using the Matched Pairs Design* Analysis of Data* *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

The Matched Pairs Design: Procedure l l Form matched pairs Randomly assign one member of each pair to the treatment condition, the other to the control condition *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Considerations in Using Matched Pairs Designs – Finding an effective matching variable – Power: A big plus – External validity l l Advantage: Don’t restrict subject population (can have heterogeneous group) Disadvantage: Results may not generalize to participants who haven’t done the matching task – Construct validity weakened because matching may tip off participants about hypothesis *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Analysis of Data in the Matched Pairs Design l l Not the between subjects t test (observations are not independent) Dependent t test: Differences between pairs/ standard error of differences *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Within-Subjects (Repeated Measures) Designs l Considerations in using within-subjects designs – Increased power – Order effects harm internal validity* *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Overview of Four Specific Sources of Order Effects l l Practice effects Fatigue effects Treatment carryover effects Sensitization *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Dealing with Order Effects l l l Minimizing each individual threat (practice, fatigue, carryover, sensitization) Use as few levels as possible to reduce opportunities for practice, fatigue, carryover, and sensitization Mixing up sequences to try to balance out order effects: Randomizing and counterbalancing *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Randomized Within-Subjects Designs l l l Procedure* Analysis of data* Summary* *Power. Point presentation

Procedure l l As in all within-subject (repeated measures) designs, each subject is observed in at least two conditions Randomly determine the sequence of treatments for each participant *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Analysis of Data l l Dependent t test (two groups) or Within-Subjects ANOVA (more than two groups) Within-subjects t (Same test as used for matched pairs analysis) – For each participant, get the difference between his/her score in Conditions 1 vs. his/her score in Condition 2. – Calculate average difference – Divide average difference by standard error of the differences – Look up t in t table under appropriate df (number of participants 1) and significance level. *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Randomized Within-Subjects Designs: Summary l l Like all within-subjects designs, quite powerful Randomization helps balance out order effects, but is there a more effective way? *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Counterbalanced Within. Subjects Designs l l Procedure* Advantages of counterbalancing* Disadvantages of counterbalancing* Conclusions about counterbalanced withinsubjects designs *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Procedure l Devise a set of sequences such that – Every condition appears in every position the same number of times (T 1 should appear first as many times as it appears last) and – Every condition precedes every other condition just as many times as it follows that condition (For every sequence in which T 1 comes before T 2, there should be a sequence in which as T 1 comes after T 2) – Example: l Sequence 1: T 1 then T 2 l Sequence 2: T 2 then T 1 l Randomly assign participants to your sequences *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Advantages of counterbalancing l l l Balance out routine order effects Learn about effect of the within-subjects variable of order (trials, position) Learn about the effect of the betweensubjects variable of sequence *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Disadvantages of counterbalancing l l May require more subjects Analysis is more sophisticated *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Counterbalanced Within-Subjects Designs: Conclusions l l Balances out routine order effects Provides information not only about the effect of treatment, but also about the effect of order (trials, position) and sequence *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Choosing an Experimental Design l l l General considerations* The two-condition case* The multiple IV case* *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

General considerations: l Pure between-subjects designs may – Have more construct validity because it is harder for participants to guess the hypothesis – Have more internal validity because they are not vulnerable to order effects – Be easier to analyze l l Within-subject designs have more power Whether within-subject designs or between-subjects designs have more external validity may depend on whether the variable is “within-subjects” or “between-subjects” in real life *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Choosing Designs: The Two Conditions Case l l Pure between subjects designs Matched pairs design Randomized within-subjects designs Counterbalanced designs *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Choosing Designs: When You Have More Than One Independent Variable l l l Using a within-subjects factorial design Using a between-subjects factorial design Using a mixed design *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;

Concluding Remarks l l l You can now intelligently choose among different types of experimental designs. You can now propose almost any type of experiment You can now read almost any write-up of a study that used an experiment *Power. Point presentation to accompany Research Design Explained © 2007 Mark Mitchell & Janina Jolley 6 th edition;