Section 4 5 Making Connections Statistics Unlocking the

Section 4. 5 Making Connections Statistics: Unlocking the Power of Data Lock 5

Today �Today we’ll see ways to simulate randomization samples for a variety of situations: Single proportion Difference in proportions Single Mean Difference in means (observational study) Correlation �Along the way we’ll make connections between… Data collection and randomization distributions Bootstrapping and randomization distributions Confidence intervals and hypothesis tests Statistics: Unlocking the Power of Data Lock 5

Generating Randomization Samples The main criteria to consider when creating randomization samples for a statistical test are: • Be consistent with the null hypothesis • Use the data in the original sample. Ideally, we also try to reflect the way the original data were collected. Statistics: Unlocking the Power of Data Lock 5

Paul the Octopus http: //www. youtube. com/watch? v=3 ESGp. RUMj 9 E Statistics: Unlocking the Power of Data Lock 5

Hypotheses • In 2008, Paul the Octopus predicted 8 World Cup games, and predicted them all correctly • Is this evidence that Paul’s chance of guessing correctly, p, is really greater than 50%? • What are the null and alternative hypotheses? H 0: p = 0. 5 Ha: p > 0. 5 Statistics: Unlocking the Power of Data Lock 5

Paul the Octopus How unusual is it to see a sample statistic as extreme as that observed, if H 0 is true? �We need to know what kinds of statistics we would observe just by random chance, if the null hypothesis were true �How could we figure this out? ? ? Simulate many samples of size n = 8 with p = 0. 5 Statistics: Unlocking the Power of Data Lock 5

Simulate! Statistics: Unlocking the Power of Data Lock 5

Lots of simulations! • For a better randomization distribution, we need many more simulations! www. lock 5 stat. com/statkey Statistics: Unlocking the Power of Data Lock 5

Randomization Distribution Statistics: Unlocking the Power of Data Lock 5

Cocaine Addiction • In a randomized experiment on treating cocaine addiction, 48 people were randomly assigned to take either Desipramine (a new drug), or Lithium (an existing drug), and then followed to see who relapsed • Question of interest: Is Desipramine better than Lithium at treating cocaine addiction? Statistics: Unlocking the Power of Data Lock 5

Cocaine Addiction • What are the null and alternative hypotheses? • What are the possible conclusions? Statistics: Unlocking the Power of Data Lock 5

Cocaine Addiction • What are the null and alternative hypotheses? • What are the possible conclusions? Reject H 0; Desipramine is better than Lithium Do not reject H 0: We cannot determine from these data whether Desipramine is better than Lithium Statistics: Unlocking the Power of Data Lock 5

R R R R R R R R R R R R 1. Randomly assign units to treatment groups Desipramine R R Lithium R R R R R R R R R R Statistics: Unlocking the Power of Data Lock 5

2. Conduct experiment 3. Observe relapse counts in each group R = Relapse N = No Relapse 1. Randomly assign units to treatment groups Desipramine Lithium R R R R N R R R R N R N N R R R N N N 10 relapse, 14 no relapse Statistics: Unlocking the Power of Data 18 relapse, 6 no relapse Lock 5

Measuring Evidence against H 0 To see if a statistic provides evidence against H 0, we need to see what kind of sample statistics we would observe, just by random chance, if H 0 were true Statistics: Unlocking the Power of Data Lock 5

Cocaine Addiction • “by random chance” means by the random assignment to the two treatment groups • “if H 0 were true” means if the two drugs were equally effective at preventing relapses (equivalently: whether a person relapses or not does not depend on which drug is taken) • Simulate what would happen just by random chance, if H 0 were true… Statistics: Unlocking the Power of Data Lock 5

R R R R N N R R R N N N N 10 relapse, 14 no relapse Statistics: Unlocking the Power of Data 18 relapse, 6 no relapse Lock 5

R R R R N N R R R N N N N Simulate another randomization Desipramine Lithium R N N N N R R R R N N N R R R N N N R R R R 16 relapse, 8 no relapse Statistics: Unlocking the Power of Data 12 relapse, 12 no relapse Lock 5

Simulate another randomization Desipramine Lithium R R R R N R R N N R R R R N R N R R N N N 17 relapse, 7 no relapse Statistics: Unlocking the Power of Data 11 relapse, 13 no relapse Lock 5

Simulate Your Own Sample �In the experiment, 28 people relapsed and 20 people did not relapse. Create cards or slips of paper with 28 “R” values and 20 “N” values. �Pool these response values together, and randomly divide them into two groups (representing Desipramine and Lithium) �Calculate your difference in proportions �Plot your statistic on the class dotplot �To create an entire randomization distribution, we simulate this process many more times with technology: Stat. Key Statistics: Unlocking the Power of Data Lock 5

P-value and Conclusion www. lock 5 stat. com/statkey p-value Using ⍺ = 0. 05: Reject H 0 We have convincing evidence that Desipramine is better than Lithium at prevent relapses in cocaine addiction. Statistics: Unlocking the Power of Data Lock 5

Choice of Statistic �By default, we often choose to use the difference in proportions when doing a randomization test for two categorical variables �However, we could just as easily use almost any other statistic (such as the difference in count of relapses in each group) and get the same result! Statistics: Unlocking the Power of Data Lock 5

Randomized Experiments • In randomized experiments the “randomness” is the random allocation to treatment groups • If the null hypothesis is true, the response values would be the same, regardless of treatment group assignment • To simulate what would happen just by random chance, if H 0 were true: o reallocate cases to treatment groups, keeping the response values the same Statistics: Unlocking the Power of Data Lock 5

Observational Studies �In observational studies, the “randomness” is random sampling from the population �To simulate what would happen, just by random chance, if H 0 were true: Simulate resampling from a population in which H 0 is true �How do we simulate resampling from a population when we only have sample data? Bootstrap! �How can we generate randomization samples for observational studies? Make H 0 true, then bootstrap! Statistics: Unlocking the Power of Data Lock 5

Body Temperatures Statistics: Unlocking the Power of Data Lock 5

Body Temperatures • In Stat. Key, when we enter the null hypothesis, this shifting is automatically done for us Stat. Key p-value = 0. 002 Statistics: Unlocking the Power of Data Lock 5

Creating Randomization Samples 1. Do males exercise more hours per week than females? 2. Is blood pressure negatively correlated with heart rate? �State null and alternative hypotheses �Devise a way to generate a randomization sample that Uses the observed sample data Makes the null hypothesis true Reflects the way the data were collected Statistics: Unlocking the Power of Data Lock 5

Exercise and Gender • H 0: m = f , Ha: m > f • To make H 0 true, we must make the means equal. One way to do this is to add 3 to every female value (there are other ways) • Bootstrap from this modified sample • In Stat. Key, the default randomization method is “reallocate groups”, but “Shift Groups” is also an option, and will do this Statistics: Unlocking the Power of Data Lock 5

Exercise and Gender p-value = 0. 095 Statistics: Unlocking the Power of Data Lock 5

Blood Pressure and Heart Rate • H 0: = 0 , Ha: < 0 • Two variables have correlation 0 if they are not associated. We can “break the association” by randomly permuting/scrambling/shuffling one of the variables • Each time we do this, we get a sample we might observe just by random chance, if there really is no correlation Statistics: Unlocking the Power of Data Lock 5

Blood Pressure and Heart Rate p-value = 0. 219 Statistics: Unlocking the Power of Data Even if blood pressure and heart rate are not correlated, we would see correlations this extreme about 22% of the time, just by random chance. Lock 5

Randomization Distribution �Paul the Octopus (single proportion): Flip a coin 8 times �Cocaine Addiction (randomized experiment): Rerandomize cases to treatment groups, keeping response values fixed �Body Temperature (single mean): Shift to make H 0 true, then bootstrap �Exercise and Gender (observational study): Shift to make H 0 true, then bootstrap �Blood Pressure and Heart Rate (correlation): Randomly permute/scramble/shuffle one variable Statistics: Unlocking the Power of Data Lock 5

Generating Randomization Samples • The specific method varies with the situation, but the general idea is always the same: simulate samples assuming the null is true • As long as the original data is used and the null hypothesis is true, most methods usually give similar answers in terms of a p-value • Technology generates the randomizations for you, so most important is not understanding how to generate randomization samples, but why (what statistics would we see if H 0 were true? ) Statistics: Unlocking the Power of Data Lock 5

Body Temperature � Statistics: Unlocking the Power of Data Lock 5

Body Temperature �We also created a randomization distribution to see if average body temperature differs from 98. 6 F by adding 0. 34 to every value to make the null true, and then resampling with replacement from this modified sample: Statistics: Unlocking the Power of Data Lock 5

Body Temperature �These two distributions are identical (up to random variation from simulation to simulation) except for the center �The bootstrap distribution is centered around the sample statistic, 98. 26, while the randomization distribution is centered around the null hypothesized value, 98. 6 �The randomization distribution is equivalent to the bootstrap distribution, but shifted over Statistics: Unlocking the Power of Data Lock 5

Bootstrap and Randomization Distributions Bootstrap Distribution Randomization Distribution Our best guess at the distribution of sample statistics, if H 0 were true Centered around the observed sample statistic Simulate sampling from the population by resampling from the original sample Centered around the null hypothesized value Simulate samples assuming H 0 were true �Big difference: a randomization distribution assumes H 0 is true, while a bootstrap distribution does not Statistics: Unlocking the Power of Data Lock 5

Which Distribution? � (a) is centered around the sample statistic, 6. 7 Statistics: Unlocking the Power of Data Lock 5

Which Distribution? � Intro stat students are surveyed, and we find that 152 out of 218 are female. Let p be the proportion of intro stat students at that university who are female. �A bootstrap distribution is generated for a confidence interval for p, and a randomization distribution is generated to see if the data provide evidence that p > 1/2. �Which distribution is the randomization distribution? (a) is centered around the null value, 1/2 Statistics: Unlocking the Power of Data Lock 5

Body Temperature Bootstrap Distribution 98. 26 Randomization Distribution H 0: = 98. 6 Ha: ≠ 98. 6 Statistics: Unlocking the Power of Data When the null value is in the tails of the bootstrap distribution (not in the CI), the sample statistic is in the tails of the 98. 6 randomization distribution (reject H 0) Lock 5

Body Temperature Bootstrap Distribution 98. 26 Randomization Distribution H 0: = 98. 4 Ha: ≠ 98. 4 Statistics: Unlocking the Power of Data 98. 4 When the null value is not in the tails of the bootstrap distribution (in the CI), the sample statistic is not in the tails of the randomization distribution (do not reject H 0) Lock 5

Intervals and Tests �A confidence interval represents the range of plausible values for the population parameter � If the null hypothesized value IS NOT within the CI, it is not a plausible value and should be rejected �If the null hypothesized value IS within the CI, it is a plausible value and should not be rejected Statistics: Unlocking the Power of Data Lock 5

Intervals and Tests If a 95% CI contains the parameter in H 0, then a two-tailed test should not reject H 0 at a 5% significance level. If a 95% CI misses the parameter in H 0, then a two-tailed test should reject H 0 at a 5% significance level. Statistics: Unlocking the Power of Data Lock 5

Body Temperatures • Using bootstrapping, we found a 95% confidence interval for the mean body temperature to be (98. 05 , 98. 47 ) • This does not contain 98. 6 , so at α = 0. 05 we would reject H 0 for the hypotheses H 0 : = 98. 6 Ha : ≠ 98. 6 Statistics: Unlocking the Power of Data Lock 5

Both Father and Mother “Does a child need both a father and a mother to grow up happily? ” Let p be the proportion of adults aged 18 -29 who say yes. • In 1997 the 95% CI for p was (0. 533, 0. 607). • In 2010 the 95% CI for p was (0. 487, 0. 573). In each case, when testing H 0: p = 0. 5 vs Ha: p ≠ 0. 5 with α = 0. 05, would we reject the null hypothesis or not? 1997: Reject H 0; 0. 5 is not within the CI, so is not a plausible value for p. 2010: Do not reject H 0; 0. 5 is within the CI, so is a plausible value for p. http: //www. pewsocialtrends. org/2011/03/09/formillennials-parenthood-trumps-marriage/#fn-7199 -1 Statistics: Unlocking the Power of Data Lock 5

Intervals and Tests �Confidence intervals are most useful when you want to estimate population parameters �Hypothesis tests and p-values are most useful when you want to test hypotheses about population parameters �Confidence intervals give you a range of plausible values; p-values quantify the strength of evidence against the null hypothesis Statistics: Unlocking the Power of Data Lock 5

Interval, Test, or Neither? Are the following questions best assessed using a confidence interval, a hypothesis test, or is statistical inference not relevant? Do a majority of adults riding a bicycle wear a helmet? On average, how much more do adults who played sports in high school exercise than adults who did not play sports in high school? Who was older, on average, the “Final Five” or the “Fierce Five” (the 2016 or 2012 USA Olympic gymnastics team)? Statistics: Unlocking the Power of Data Lock 5

Summary �Randomization samples should be generated Consistent with the null hypothesis Using the observed data Reflecting the way the data were collected �If a null hypothesized value lies inside a 95% CI, a two-tailed test using α = 0. 05 would not reject H 0 �If a null hypothesized value lies outside a 95% CI, a two-tailed test using α = 0. 05 would reject H 0 Statistics: Unlocking the Power of Data Lock 5