8 Experimental Research Design Introduction n Research design

+ 8 Experimental Research Design

+ Introduction n Research design n the outline, plan, or strategy used to investigate the research problem n Purpose of research design n control for unwanted variation n suggests how data will be statistically analyzed n Goal of research design n choose the strongest design that is possible, ethical, and feasible

+ Strong Experimental Designs n Designs that effectively control extraneous variables and provide strong evidence of cause and effect n Improved internal validity achieved by eliminating rival hypotheses n n n with control techniques n most important is random assignment with a control group n group that does not get the independent variable or gets some standard value n serves as source of comparison to experimental group n controls for rival hypothesis Experimental group n the group of participants that receives the treatment condition that is intended to produce an effect

+ Between-Participants Designs n Different groups are exposed to the different levels of the independent variable n participants are randomly assigned to groups n Posttest-only control-group design n random assignment to groups creates equivalence n use of control group eliminates most threats to internal validity n weaknesses of design n n does not guarantee equivalence of groups – particularly with small sample size (N < 30) no pretest to assess equivalence

Posttest-Only Control-Group Design n Example n participants randomly assigned to groups n IV = Social Skills Training n n experimental group – received social skills training (X) control group – receives no social skills training n DV = posttest measure (O) for both groups

+ Posttest-Only Control-Group Design with Three Groups

+ Figure 8. 6 Pretest-posttest control-group design.

+ Between-Participants Designs n Pretest-Posttest (cont'd) Control-Group Design n pretest added to posttest-only control-group design n advantages of including a pretest n n can assess the effects of randomization – insure that groups are equivalent on dependent variable prior to introduction of experimental conditions n can assess the effects of additional variables that may interact with independent variable n determine if ceiling effect has occurred n allows use of analysis of covariance to statistically control for pretest differences n allows researcher to assess the change in dependent variable from pretest to posttest potential weakness – may not generalize to situations with no pretest

+ Within-Participants Designs http: //explorable. com/repeated-measures-design n Participants included in all conditions (also known as repeated measures designs) n Counterbalancing necessary to eliminate linear sequencing effects n Within-participants posttest-only design n all participants receive all conditions, and a posttest is administered after each condition is administered

+ Within-Participants Posttest-Only Design n Example (Mahoney, et al. , 2005) n participants = elementary school children n IV = breakfast type n cereal, oatmeal, no breakfast n each type on three different days n DV = cognitive performance n series of cognitive tasks after breakfas

+ Within-Participants Designs n Strengths n increased sensitivity because effects of individual differences are controlled n fewer research participants needed n Weaknesses n difficult for participants n potential problem of differential carryover effects

+ Mixed Designs n Contains both between participants and within participants variables n Pretest-Posttest Control-Group Design n pretest added to posttest-only control-group design n between participants variable = variable of interest n within participants variable = time (pretest and posttest)

+ Pretest-Posttest Control-Group Design

+ Pretest-Posttest Control-Group Design n Pretest for social anxiety n Random assignment of 100 participants with social anxiety n IV = treatment for social anxiety n n experimental group = anxiety reduction treatment (N = 50) control group = no anxiety reduction treatment (N = 50) n DV = level of social anxiety (posttest) n Results

+ Mixed Designs n Pretest-posttest n advantages of including a pretest n n n control-group design can assess the effects of randomization n insure that groups are equivalent on dependent variable prior to introduction of experimental conditions determine if ceiling or floor effect has occurred allows use of analysis of covariance to statistically control for pretest differences allows researcher to assess the change in dependent variable from pretest to posttest Disadvantage of including a pretest n may not generalize to situations with no pretest

+ Factorial Designs n A design that includes two or more IVS to determine their separate (main effects) and joint effects (interaction) on DV n. Purpose: Look at multiple factors and see how they interact (the joint effect; is there an interaction between Education & Salary? n. Variable (or Factor): Any measurable behavior, attribute or characteristic (e. g. , Age, height, ethnicity) n They can include: n only between participants variables n only within participants variables n both between and within participants variables (mixed design)

Factorial Designs n. Variable n. Levels (or Factor): Age, height, ethnicity of Factor: n Age: Old vs. Young <------- Two levels n Height: Short vs. Tall n Ethnicity: Asian vs. Hispanic

+ Factorial Design Layout Example Caffeine Consumption low Sleep Deprivation medium high low medium high not not deprived deprived low medium high deprived

+ Factorial Design n Design layout and analysis n cell n combination of levels of two or more independent variables n cell mean n the average score of the participants in a single cell n marginal n the mean average score of all participants receiving one level of an independent variable

+ Factorial Design Layout with Analysis

+ Factorial Designs n Main effect n the influence of one independent variable on the dependent variable n ignoring the second IV n one main effect for each IV in a study n Interaction n the effect joint, combined, or “interactive” effect of two or more independent variables on the dependent variable n i. e. , when the effect of one independent variable depends on another n when displayed graphically, an interaction yields non-parallel lines

Factorial Design Example n DV = driving performance or number of correct maneuvers n Main n not deprived = more correct maneuvers n Main n effect of sleep deprivation effect of caffeine medium caffeine consumption most correct maneuvers

Line Graph of Interaction n Interaction effect between sleep deprivation and caffeine consumption n gives us more information than main effects if deprived of sleep, the more caffeine the better when it comes to driving performance if not deprived of sleep, medium levels of caffeine consumption increases deriving performance the most

+ Factorial Design Notation n 2 x 2 design n number of numerals = number of IVs = 2 n each number indicates the number of levels for each IV n IV 1 = 2 levels n IV 2 = 2 levels n 2 x 3 design n 2 IVs n IV 1 = 2 levels n IV 2 = 3 levels

2 -way Design Variable Gender has two levels n 2 (Gender: M, F) x 2 (Class: Math vs. Eng) Math Gender Male Female Variable Class has two levels Class English Matrix has 4 cells 2 X 2=4

+ Within Participants Factorial Design

+ Mixed Model Factorial Design

+ Strengths and Weaknesses of Factorial Designs n Strengths n more than one independent variable allows for more precise hypotheses n control of extraneous variables by including as an independent variable n ability to determine the interactive effect of two or more independent variables

+ Strengths and Weaknesses of Factorial Designs n Weaknesses n using more than two independent variables may be logistically cumbersome n examples n n n 2 x 2 design = 4 cells, 2 main effects, and 1 interaction 2 x 3 design = 6 cells, 2 main effects, and 1 interaction 2 x 3 design = 12 cells, 3 main effects, and 4 interactions n higher-order interpret interactions are difficult to

+ Choice/Construction of the Appropriate Experimental Design n Examination of prior research literature can guide choice of design n Many factors to consider n use of control group n number of comparison groups n pretest(s) n within-participants or between-participants n number of independent and dependent variables

+ Strengths and Weaknesses of Factorial Designs (cont'd) n Disadvantages of factorial designs n using more than two independent variables may be logistically cumbersome n higher-order interactions are difficult to interpret

+ Choice/Construction of the Appropriate Experimental Design n Examination of prior research literature can guide choice of design n Many factors to consider n use of control group n number of comparison groups n pretest(s) n within-participants or between-participants n number of independent and dependent variables�

Three-Factor n 2 (Gender: M, F) X 2 (Class: Eng, Math) X 2 (Ethnicity: Anglo, Mex) n. Factor 1 (Gender): Two levels (M, F) n. Factor 2 (Class): Two levels (Eng, Math) n. Factor 3 (Ethnicity): Two levels (Anglo, Mexican) n. How n 2 many possible cells? X 2 = 8 Cells

+ A 2 x 3 Example n Factor 1: Treatment n Psychotherapy n Behavior n modification Factor 2: Setting n Inpatient n Day treatment n Outpatient

+ A 2 x 3 Example Treatment In Setting Day Out Psych 5 7 9 7 Behmod 5 7 9 7 5 7 9 Main Effect of Setting

+ A 2 x 3 Example Treatment In Setting Day Out Psych 5 5 Behmod 7 7 6 6 6 Main Effect of Treatment

+ A 2 x 3 Example Treatment In Setting Day Out Psych 5 6 7 6 Behmod 7 6 5 6 6 Interaction Effect

+ A 2 x 3 Example Treatment In Setting Day Out Psych 5 7 6 6 Behmod 7 8 6 7. 5 6 Interaction Effect

Research Methods and Statistics: A Critical Thinking Approach, Third Edition by Sherri L. Jackson Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

Research Methods and Statistics: A Critical Thinking Approach, Third Edition by Sherri L. Jackson Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

Research Methods and Statistics: A Critical Thinking Approach, Third Edition by Sherri L. Jackson Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

Research Methods and Statistics: A Critical Thinking Approach, Third Edition by Sherri L. Jackson Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

Research Methods and Statistics: A Critical Thinking Approach, Third Edition by Sherri L. Jackson Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

Research Methods and Statistics: A Critical Thinking Approach, Third Edition by Sherri L. Jackson Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

+ n Factorial Designs (cont'd) Mixed model factorial designs n uses a combination of within-participants and between-participants n WITHIN GROUPS: PRE-TEST n BETWEEN GROUPS: TYPE OF THERAPY

Diff = 147 * Diff = 38 * Figure 1. Reaction Time (in Ms) as a Function of Language and Relatedness for Standard Spoken Spanish

Diff = 85 * Diff = 23 * Figure 2. Reaction Time (in Ms) as a Function of Language and Relatedness for Accented Spoken Spanish

Figure 3: 2 -way Interaction (Language X

Figure 4: 2 -way Interaction (Accentedness X