Research Methods Design and Analysis Twelfth Edition CHAPTER

Research Methods, Design, and Analysis Twelfth Edition CHAPTER 7 Control Techniques in Experimental Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Introduction • Points to note 1. random selection rarely used – researchers focus on the independent variable 2. purposive and convenience samples typically used 3. external validity increased by replication • For the rest of this chapter, assume the researcher already has a sample and our focus will be on control techniques Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Introduction Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Goal of Experimentation • Identify the causal effect of the independent variable (IV) on the dependent variable (DV) – must have internal validity to do this § requires control of extraneous variables § eliminate differential influence – confounds Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Random Assignment • A control technique to equate groups of participants – accomplished by ensuring that every member has an equal chance of being assigned to any group • Provides maximum insurance that groups are equal • Eliminates systematic differences between groups • Doesn’t eliminate extraneous variables, but randomly distributes them across groups • The most important and powerful control technique • www. randomizer. org Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Random Assignment Example Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Matching • Use of any of a variety of techniques to equate participants in the treatment groups on specific variables – should be done with variables thought to be related to the IV or may confound IV – e. g. , intelligence, age, gender • Advantages of matching – controls for the variables on which participants are matched – increases the sensitivity of the experiment Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Matching Techniques • Holding variables constant – example § gender and conformity study – using only females – disadvantages § restricts the population size § restricts generalization to the type of participants in the study Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Matching Techniques • Building the extraneous variable into the research design (blocking) – make the extraneous variable another IV in the study – should be used only when you are interested in the effect of the extraneous variable Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Matching Techniques • Yoked control – matches participants on the basis of the temporal sequence of administering an event – Example § research question – a researcher wants to know the effect of students having the freedom to choose snack breaks on their classroom productivity § each control participant is “yoked” to an experimental participant § experimental participant takes a self-chosen snack break § the control “yoked” participant is given a snack break by the teacher Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Matching Techniques • Matching by equating participants – individual matching § match case by case then randomly assign to groups § advantage – groups equated on potential extraneous variable § disadvantages – identifying the variables on which to match – matching participants increases as the number of variables on which to match increases – some variables difficult to match – decrease in generalizability of results Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Individual Matching Example Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Individual Matching Example Name IQ Sara Jane Donald Debbie Jake Jody Kathy Brian 110 95 120 105 95 110 Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Random Assignment to conditions: Treatment Control Brian Sara Jane Kathy Debbie Jody Jake Donald Average IQ by group: Treatment Control 107. 5 Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Counterbalancing • Used only with repeated measures (within participants) designs • All participants receive all treatment conditions Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Counterbalancing • Used to control sequencing effects – two types of sequencing effects § order effects – a sequencing effect arising from the order in which the treatment conditions are administered to participants • e. g. , practice, fatigue § carryover effects – a sequencing effect that occurs when performance in one treatment condition affects performance in another treatment condition • e. g. , effects of treatment A still affecting participant when engaged in treatment B Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Counterbalancing • Counterbalancing procedures – randomized counterbalancing § sequence of conditions is randomly determined for each participant § example: – – IV with three levels six possible sequences (ABC, ACB, BAC, BCA, CAB, CBA) each participant randomly assigned to one sequence can use randomizer. org or a random table of numbers Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Counterbalancing • Counterbalancing procedures – intrasubject counterbalancing § participants take treatments in more than one order § may not be feasible with long treatment sequences § example: – IV with three levels – six possible sequences (ABC, ACB, BAC, BCA, CAB, CBA) – each participant is exposed to all sequences Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Counterbalancing • Counterbalancing procedures – complete counterbalancing § § all possible sequences of treatment conditions are used participants randomly assigned to sequence N! = N multiplied by each number below it examples: – 2! = 2 x 1 = 2 • AB, BA – 3! = 3 x 2 x 1 = 6 • ABC, ACB, BAC, BCA, CAB, CBA – 4! = 4 x 3 x 2 x 1 = 24 § rarely used with more than 3 conditions because number of possible sequences (N!) is too large Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Counterbalancing • Counterbalancing procedures – incomplete counterbalancing § most commonly used technique § not all possible sequences are used § criteria – for sequences enumerated – each treatment condition must appear an equal number of times in each ordinal position and – each treatment condition must precede and be followed by every other condition an equal number of times § sequences determined by formula – 1, 2, n, 3, (n-1), 4, (n- 2), 5 Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Incomplete Counterbalancing Example Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Counterbalancing • Differential carryover – a treatment condition affects participants’ performance in a later condition in one way and in another way when followed by a different condition – example § the carryover from treatment A is 4 units when it is immediately followed by treatment B, but the carryover from treatment A is 2 units when it is immediately followed by treatment C – if suspected, don’t use repeated measures Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Control of Participant Effects • Double-blind placebo method – neither the experimenter nor the research participant is aware of the treatment condition administered to the participant • Deception – omission of or altering the truth of information given to the participant during a research study – used when there is no other way to gain the knowledge and risk does not outweigh the benefit of the information – must keep the false information constant for all participants Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Control of Participant Effects • Control of participant interpretation – techniques used to gain insight into participants’ perceptions of experiment – retrospective verbal report § an oral report in which the participant retrospectively recalls aspects of the experiment § questions to ask – – what did the participant think the experiment was about? what did he or she think the experimenter expected to find? what type of response did the participant attempt to give, and why? how does the participant think others will respond in this situation? Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Control of Participant Effects • Control of participant interpretation – concurrent verbal reports § a participant’s oral report of the experiment, which is obtained as the experiment is being performed § sacrifice groups – groups of participants who are stopped and interviewed at different stages of the experiment § concurrent probing – obtaining a participant’s perceptions of the experiment after completion of each trial § think-aloud technique – a method that requires participants to verbalize their thoughts as they are performing the experiment Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Control of Experimenter Effects • Control of recording errors – make researchers aware of making careful observations § may require training of researcher before data collection – use multiple data recorders (e. g. , video cameras, computers) or observers – have participants make responses on a computer • Control of attribute errors – referring to attributes of the experimenter – use the same experimenter in all treatment conditions unless the treatment condition interacts with attributes Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.

Control of Experimenter Expectancy • Control of experimenter expectancies – blind technique § a method whereby knowledge of each research participant’s treatment condition is kept from the experimenter § e. g. , double-blind procedure – partial blind technique § a method whereby knowledge of each research participant’s treatment condition is kept from the experimenter through as many stages of the experiment as possible – automation § the technique of totally automating the experimental procedures, so that no experimenter-participant interaction is required Research Methods, Design, and Analysis, Twelfth Edition Christensen • Johnson • Turner Copyright © 2014 by Pearson Education, Inc. All rights reserved.