Chapter 2 Experimental Design Observational Study vs Experiment

Chapter 2 Experimental Design

Observational Study vs. Experiment Observational Study: Observe outcomes without imposing any treatment Experiment: Actively impose some treatment in order to observe the response

r u f s e k a Hippity Hop. I’ve developed a new rabbit Mfood, & t f so y! n i h s s e s a e r c n ! I y g its r e en rabb d s n a p l He larger ! r w gro tronge s ood Ra F bbit

Can I just make these claims? NO What must I do to make these claims? Do an experiment Who/what should I test this on? Rabbits What variable am I testing? Type of food

• Experimental Units: The individuals (people, animals, plants, etc. ) receiving the treatments • Factor: The explanatory variable • Level: A specific value of the factor

• Response Variable: What you measure • Treatment: A specific experimental condition applied to the units

I plan to test my new rabbit food. What are my experimental units? Rabbits What is my factor? Type of food What is the response variable? How well the rabbits grow

I’ll use my pet rabbit, Lucky! Hippity Hop Since Lucky is big and strong, Hippity Hop is a better rabbit food!

• Control Group: Something to compare the factor against Can be the old/current item or a placebo • Placebo: A “dummy” treatment with no intended effect

Old Food Hippity Hop Now I’ll use Lucky & my friend’s rabbit, Flash. Lucky gets Hippity Hop food and Flash gets the old rabbit food. WOW! Lucky is bigger and stronger, so Hippity Hop is better!

Old Food Hippity Hop The first five rabbits I catch will get Hippity Hop food, and the remaining five will get the old food. The Hippity Hop rabbits are bigger, so it’s the better food!

Old Food Hippity Hop Number thethe rabbits fromin 1 a– hat. 10. Place numbers Theremaining first five numbers pulled The rabbits get the old from the hat will be the rabbits food. that get Hippity Hop food. 98573 6 2 4 rabbits & found that the I evaluated the 5 9 better 10 rabbits eating Hippity Hop are 1 7 3 8 than the old food!

• Blinding: Units do not know which treatment they are getting • Double Blind: Neither the units nor the evaluator know which treatment each subject received

i d p o p i H bit Fo ft o b s a r R u t f i s d e n k a ma iny, wth h s o r d g f an ases o s e e p r y c t in L L ! A s t r fo rabbi e ood Ra F bbit ak m I n ? a m i C cla s i th

Principles of Experimental Design • Control effects of extraneous variables Keep conditions consistent Compare treatment groups to a control group (placebo or “old”) • Replication of the experiment on many subjects to eliminate natural variation • Randomization: Use chance to assign subjects to treatments

The ONLY way to show cause & effect is with a welldesigned experiment!

Ex. 1: A consumer group wants to test cake pans to see which bakes the most evenly. It will test aluminum, glass, and plastic pans in both gas and electric ovens. Experimental units? Cake batter Factors? Type of pan, Type of oven Levels? Type of Pan: aluminum, glass, plastic Type of Oven: electric and gas Response variable? Number of treatments? How evenly the cake bakes 6

Ex. 2: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Experimental units? Plots of land Factors? Type of fertilizer Levels? The 3 types Response variable? Yield of crop How many treatments? 3

Ex. 2: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Why is the same type of seed used on all 15 plots? Control – Otherwise, type of seed would be another factor What are other potential extraneous variables? Type of soil, amount of water, etc. Does this experiment have a placebo? No – there is built-in comparison, so a placebo is not needed

Experimental Designs Random assignment Completely Randomized Design: All experimental units are assigned at random among the treatments

Treatment A Treatment B Treatment C Treatment D Randomly assign experimental units to treatments Completely randomized design

Random assignment Randomized Block Design: Units are blocked into homogeneous groups, then randomly assigned to treatments within each block Block units on a variable that might affect the response

Treatment A Treatment B Put into homogeneous Randomly assign groups experimental units to treatments

Confounding Variable: A variable whose effect cannot be separated from the effects of the explanatory variable (factor) Suppose we want to test a new deodorant against one already on the market. What variables might affect the effectiveness of a deodorant?

Treatment A Treatment B If women get better results, is it due to a better deodorant? Treatment and gender are confounded Treatment B Confounding does NOT occur in a completely randomized design! Treatment A

Matched Pairs Design • Special type of blocked design Match up units by a similar characteristic Randomly assign one to one treatment The other gets the 2 nd treatment OR Have each unit do both treatments in random order (e. g. Pepsi Challenge) Each person is their own matched pair • Assignment of treatments is dependent

Treatment A Treatment B experimental units Next, Pair randomly assign one according to specific unit from a pair to characteristics. Treatment A. The other unit gets Treatment B. Compare results within each pair – do not compare pairs to each other, since they have different characteristics

Ex. 3: An article from USA Today reports the number of victims of violent crimes per 1000 people. 51 victims have never been married, 42 are divorced or separated, 13 are married, and 8 are widowed. Is this an experiment? Why or why not? No – no treatment was imposed on people. What is a potential confounding variable? Age – Young people are more likely to have never been married and more at risk of violent crimes

Ex. 4: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly assigned to one of the four programs, and their speeds are measured. Is this an experiment? Why or why not? Yes, treatments are imposed What type of design is this? Completely randomized Factors? Levels? Word-processing program; 4 levels Response variable? Speed

Ex. 4: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly designed to one of the four programs and their speeds are measured. Is there a potential confounding variable? No – completely randomized designs have no confounding Can this design be improved? Lots of variation in typing speed Block design: Each person uses all 4 programs in random order Each person is their own block

Ex. 5: Suppose a manufacturer wants to test a new fertilizer against the current one on the market. Ten 2 -acre plots of land scattered throughout the county are used. Each plot is divided into two subplots; one subplot is treated with the current fertilizer, and the other is treated with the new fertilizer. Wheat is planted and the crop yields are measured. What type of design is this? Matched Pairs Design Why use this method? Reduces the effects of variation in plots Where does randomization occur in the design? Randomly assign a fertilizer to first subplot in each plot

The Two Enemies of Statistics: Bias & Variability High bias High variability Low bias Low variability

Reducing Bias: • Randomization spreads uncontrolled variables evenly throughout the treatment groups Reducing Variability: • Blocking units by common characteristics • Larger sample size helps cancel out variability