Chapter 13 Experiments and Observational Studies Madison Vincent

+ Chapter 13: Experiments and Observational Studies Madison Vincent Period: 3 12/11/14

+ Observational Studies ■ In observational studies researchers don’t assign choices; they simply observe them ■ Based on data in which no manipulation of factors has been employed. ■ Valuable for discovering trends and possible relationships ■ Not possible for observational studies to demonstrate a casual relationship

+ Retrospective and Prospective Study Retrospective Subjects are selected and then their previous conditions or behaviors are studied. Are not based on random samples so they usually focus on estimating differences between groups or associations between variables. Prospective Subjects are followed to observe future outcomes No treatments are deliberately applied so it is not an experiment Typically focus on estimating differences among groups that might appear as the groups are followed during the course of the study

+ Experiments ■ Study design that allows us to prove a cause-and-effect relationship Manipulates factor levels to create treatments, Randomly assigns subjects to these treatment levels, Then compares the responses of the subject groups across treatment levels Factors are variables whose levels are controlled by the experimenter. Experiments attempts to discover the effects that differences in factor levels may have on the responses of the experimental units Response is variables whose values are compared across different treatments. In a randomized experiment, large response differences can be attributed to the effect of differences in treatment level

+ Experiments ■ In an experiment, the experimenter actively and deliberately manipulates the factors to control the details of the possible treatments, and assigns the subjects to those treatments at random. ■ The experimenter then observes the response variable and compares responses for different groups of subjects who have been treated differently. ■ The best experiments are usually randomized, comparative, double blind, and placebo controlled.

+ Experiments ■ Experimental units are individuals on whom an experiment is preformed. Usually called subjects or participants when they are human ■ Level is the specific values that the experimenter chooses for a factor are called the levels of the factor ■ Treatment is the process, intervention, or other controlled circumstance applied to randomly assigned experimental units. Treatments are the different levels of a single factor or are made up of combinations of levels of two or more factors

+ Four Principles of Experimental Design ■ 1) Control aspects of the experiment that we know may have an effect on the response, but that are no the factors of being studied ■ 2) Randomize subjects to treatments to even out effects that we cannot control ■ 3) Replicate over as many subjects as possible. Results for a single subject are just anecdotes. If the subjects of the experiment are not a representative sample from the population of interest, replicate the entire study with a different part of the population ■ 4) Block to reduce the effects of identifiable attributes of the subjects, preferably controlled

+ Diagrams and Statistically Significant Diagrams ■ It is often helpful to diagram the procedure of an experiment Statistically Significant When an observed difference is too large for us to believe that it is likely to have occurred naturally, we consider the differences to be statistically significant. Subsequent chapters will show specific calculations and give rules, but the principle remains the same

+ Control Treatments ■ A baseline (“business as usual”) measurement is called a control treatment ■ Control Group – The experimental units assigned to a baseline treatment level, typically either the default treatment, which is well understood, or a null, placebo treatment. Their responses provide a basis for comparison

+ Blinding ■ Blinding – Any individual associated with an experiment who is not aware of how subjects have been allocated to treatment groups is said to be blind ■ There are 2 main classes of individuals who can affect the outcome of an experiment: those who could influence the results and those who evaluate the results Single-Blind When every individual in either of these cases is blinded Double-Blind When everyone in both classes is blinded

+ Placebos ■ Placebo – A treatment known to have no effect, administered so that all groups experience the same conditions. Many subjects respond to such treatment( a response known as the placebo effect). Only by comparing with a placebo can we be sure that the observed effect of a treatment is not due simply to the placebo effect. ■ Placebo effect – The tendency of man human subjects (often 20% or more of experiment subjects) to show a response even when administered a placebo

+ Blocking ■ Block – When groups of experimental units are similar, it is often a good idea to gather them together into blocks. By blocking we isolate the variability attributes to the differences between the blocks so that we can see the differences caused by the variables of interest. ■ Designs – In a randomized block design, the randomization occurs within blocks. In a completely randomized design, all experimental units have an equal chance of receiving any treatment. ■ Matching – In a retrospective or prospective study, subjects who are similar in ways not under study may be matched and then compared with each other on the variables of interest. Matching, like blocking, reduces unwanted variation

+ Confounding and Lurking Variable ■ Confounding – When the levels of one factor are associated with the levels of another factor so their effects cannot be separated, we say that these two factors are confounded ■ A lurking variable creates an association between two other variables that tempts us to think that one may cause the other.

+ Question #29 ■ Will listening to a Mozart piano sonata make you smarter? In a 1995 study, Rauscher, Shaw, and Ky reported that when students were given a spatial reasoning section of a standard IQ test, those who listened to Mozart for 10 minutes improved their scores more than those who simply sat quietly. A) These researchers said the differences were statistically significant. Explain what that means in this context. The differences among Mozart and quiet groups were more than would have been expected from ordinary sampling B) Steele, Bass, and Crook tried to repicate the origional study. The subjects were 125 college students who participated in the experiment for course credit. Subjects firsttook the test. Then they were assigned to one of the three groups: listening to Mozart piano sonata, listening to music by Philip Glass, and sitting for 10 min in sielnce. Three days after the treatments, they were retested. Draw a diagram displaying the design of this experiment

+ #27 Music by Glass Pretest Mozart piano sonata Post Test Silence C) The box plots on p. 315 show the differences in score before and after treatment for the three groups. Did the Mozart group show improvement? The Mozart group seemed to have the smallest median difference and thus the least improvement, but there does not appear to be a significant difference. D) Do you think the results prove that listening to Mozart is beneficial? No, if anything there is less improvement, but the difference does not seem significant compared with the usual variation

+ Question #29 ■ The makers of Frumpies, “the breakfast of rug rats, ” want to improve their marketing, so they consult you: A) They first want to know what fraction of children ages 10 to 13, like their celery-flavored cereal. What kind of study should they preform Observational. Randomly select a group of children, ages 10 to 13, have them taste the cereal, and ask if they like the cereal B) They are thinking of introducing a new flavor, maple-marshmallow Frumpies, and want to know whether children will prefer the new flavor to the old one. Design a completely randomized experiment to investigate this question Get volunteers ages 10 to 13. Each volunteer will taste both cereals, randomizing the order in which they taste them. Compare the percentage of favorable ratings for each cereal.

+ #29 ■ They suspect that children who regularly watch the Saturday morning cartoon show starring Frump, the flying teenage warrior rabbit who eats Frumpies in every episode, may respond differently to the new flavor. How would you take that into account in your design? From volunteers, identify the children who watch Frump and identify the children who do not watch Frump. Use a blocked design to reduce variation in cereal preference that may be associated with watching the Frump cartoon