Experimental Design Negar Hariri Introduction The fundamental principle

Experimental Design Negar Hariri

Introduction The fundamental principle of science, the definition almost, is this: the sole test of the validity of any idea is experiment. – Richard P. Feynman Good experimental design allows you to: • Isolate effects of each input variable • Determine effects due to interactions of input variables • Determine magnitude of experimental error • Obtain maximum info with minimum effort 2

Organizing Your Experiment Before you can write the steps of a procedure, you must identify: Independent variable Dependent variable Constant variables (controlled variables) The control (most of the time) 3

Experimental Design If X Y & If NOT X Strong internal validity NOT Y 4

Two-Group Experimental Designs Two "equivalent" groups are created üProbabilistically equivalent üAssign people randomly from a common pool of people into the two groups. One group (the treatment group) gets the program and other group (the comparison or control group) does not. 5

What is Probabilistic Equivalence? 6

Two-Group Experimental Designs The groups are compared by testing for the differences between the means using at-test or other methods. Properties üPretest is not required. üStrong against the single-group threats to internal validity. üIt is susceptible to all of the social interaction threats to internal validity. 7

Random Selection & Assignment Random selection is how you draw the sample of people for your study from a population. ü Random selection is related to sampling. Therefore it is most related to the external validity of your results. Random assignment is how you assign the sample that you draw to different groups or treatments in your study. ü Random assignment is most related to internal validity. 8

Classifying Experimental Designs 9

Classifying Experimental Designs Signal enhancers Experimental designs Noise-reducing experimental designs Factorial designs Covariance designs Blocking designs 10

Factorial Design 11

The Null Case 12

The Main Effect Of Time 13

The Main Effect 14

Interaction Effects 15

Factorial Design Variations Dependent measure: severity of illness rating done by the treatment staff. . Factor 1: Treatment ü psychotherapy ü behavior modification Factor 2: Setting ü inpatient ü day treatment ü outpatient 16

Factorial Design Variations 17

Randomized Block Designs Goal ü Reducing noise in the data What are the steps? ü Divide the sample into relatively homogeneous subgroups or blocks ü The experimental design you want to implement is implemented within each block or homogeneous subgroup The variability within each block should be less than the variability of the entire sample. What if the subgroups are not homogeneous? 18

Randomized Block Designs 19

How Blocking Reduces Noise 20

Covariance Designs Another method for reducing noise 21

Covariance Designs 22

Covariance Designs 23

Covariance Designs 24

Covariance Designs 25

Hybrid Experimental Designs Mitigate specific threats to internal validity Combine advantages of different standard designs Example? ü Switching Replications Design 26

Switching Replications Design Two Groups The two groups switch roles so all participants have received the treatment 27

Switching Replications Design 28