Hypothesis testing Summer Program Brian Healy Last class

Hypothesis testing Summer Program Brian Healy

Last class n Study design – What is sampling variability? – How does our sample effect the questions we can answer? n Basics of probability n Central limit theorem n Sample mean

What are we doing today? n Rare event n p-value n Hypothesis test n t-distribution / sample standard deviation

Big picture We discussed last week that we could estimate the population mean with the sample mean and the central limit theorem told us the distribution of the sample mean. n Now, we are going to consider testing whether or not our sample mean is equal to a hypothesized value. We call this hypothesized value the null hypothesis. This test allows us to compare our sample to a value in a statistically meaningful way. n

Null hypothesis n n We set up our null hypothesis so that we can reject the null hypothesis. The test is designed to disprove the null The first and most important step in any problem. This part requires knowledge of the problem. Notation: H 0: My mother can run a 5 minute mile. n H 0: The probability of heads on the coin is 0. 5. n n – Not: My mother cannot run a 5 minute mile. – Not: The probability is not 0. 5

Alternative hypothesis n Notation: HA or H 1 n Has two characteristics – Must cover all values not included in the null – Must contain the value that we think is going to happen n HA : My mother runs a mile slower than 5 minutes n HA: The probability of heads is not 0. 5

Hypothesis test Definition: A statistical test of a null hypothesis n Completed under the assumption that the null is true (conditional probability) n Always want to disprove the null hypothesis n – – n Ex. H 0: Mom’s mean time<=5: 00 One-sided HA: Mom’s mean time>5: 00 Alternatively: H 0: Probability of heads=0. 5 Two-sided HA: Probability of heads != 0. 5 The most important step is properly defining the null and alternative hypotheses

How do we test this hypothesis? Take a sample n As we have discussed, we want to think carefully about the how to collect the sample to ensure that we limit bias confounding and allow the results to be generalized to the proper population. n From this sample, we can find a summary statistic and compare this to null hypothesis n – Mean (t-test, linear regression) – Median (Wilcoxon tests, quantile regression)

What does this have to do with the CLT? n To test a hypothesis, we take a sample and find the sample mean – Ex. Have my mom run a mile 10 times, or flip the coin 20 times – Determining the proper sample size is next class Under the null hypothesis, we know the population mean n We sometimes may know the population variance n The distribution of the sample mean is normal with known mean and variance under these conditions n

Distribution of test statistic n Under the null hypothesis, we know that the distribution of is normal with mean m and standard deviation Now, we want to find the probability of observing the sample mean or a value more extreme, under the null (p-value) to see if the null hypothesis is likely true or false. n Have we observed a rare event? Is it rare enough to reconsider the null? n

What is a rare event? n n My mom claims that she runs a mile in 5 minutes. I think she can’t How can I test this? What happens if she ran a mile in – 5: 15 minutes? – 6 minutes? – 10 minutes? n What if she ran 5 separate miles at 10 minutes on average?

What is a rare event? n n n You play a game against a friend. In this game, you win a dollar if the coin is heads and you lose a dollar if the coin is tails What is the null hypothesis? What if the coin landed on tails 2 consecutive times? What if the coin landed on tails 10 consecutive times? At what point would you start to get suspicious? We want to know if the event we observed could have happened simply by chance or if something else is more likely going on

P-value n n Tells you how rare the event is Definition: Given a null hypothesis, the probability of the observed value or something more extreme P(event or something more extreme | Ho is true) Ex. Coin toss problem – Null hypothesis: P(tails)=0. 5 – Sample 9 out of 10 tails – P(9 or more tails | H 0 is true)=P(9 tails | H 0 is true)+P(10 tails | H 0 is true)=0. 011

Alpha level-type I error n n n Definition: probability of rejecting the null hypothesis when the null hypothesis is in fact true (rejection probability). Usually 0. 05 or 0. 1, but set by the investigator Compare the p-value to the alpha level to determine if you have a significant result. This value defines how rare an event needs to be for use to say that the event did not occur by chance. It is called an error because this conclusion that the result was not due to chance is wrong a*100% of the time. One-sided or two-sided

Steps for hypothesis testing 1) 2) 3) 4) 5) State null and alternative hypotheses State type of test and alpha level Determine and calculate appropriate test statistic Calculate p-value Decide whether to reject or not reject the null hypothesis • 6) NEVER accept null Write conclusion

Example A study in New Bedford was looking at pregnant teens to see how long after pregnancy did each young woman arrive at the physician’s office for the first visit and the amount of time between the first visit and the second visit. n Questions: Do teens from a low income area arrive at a clinic later than the average woman? Is there more time between the first and second visit among these teens? n

It is known that the average amount of time from conception until a woman first visits her doctor is 8. 5 weeks (this number is an estimate because it is difficult to know exactly when conception occurred) and the average amount of time from first visit to second visit is 4. 3 weeks. n For the moment, let’s assume that we know the population standard deviations for each of these are 2. 6 weeks and 2. 2 weeks, respectively. n We have collected a sample of 35 pregnant teens and we would like to know if they take longer to get their first visit than the average woman n

Sample data As with all of the data sets from now on, the data is on the BIO 232 website. n Let’s determine the mean for this sample and compare it to the hypothesized value. n preg<-read. table(“preg. dat”, header=T) first<-preg[, 1] mean(first) #This is the sample mean [1] 9. 74 n So the sample mean is clearly not equal to the population mean (8. 5 weeks), but is it sufficiently different to say that these girls are different than the population. n

Steps for hypothesis testing Null: m=8. 5 weeks, Alternative: m != 8. 5 weeks 2) One sample hypothesis test, alpha=0. 05 1) 3) Area in upper tail = 0. 0024, p-value = 0. 0048 5) Reject null 6) Conclusion: There is a difference in the amount of time from conception to the first visit to a physician. The time is longer for the pregnant teens. 4)

Picture Area=0. 0024 n Here is a picture Area=0. 0024 8. 5 9. 74

Normal hypothesis test in R n To complete a normal hypothesis test in R, you can simply use the pnorm command with the appropriate mean and standard deviation. Remember, pnorm provides the area in the lower tail in all cases n For the previous problem, to get the appropriate 2 -sided p-value, use (1 -pnorm(9. 74, 8. 5, 2. 6))*2

Another way to look at the test Given a specific alpha level, you can find the cut -off for which all values more extreme, the null hypothesis would be rejected n The region more extreme is called the rejection region n For our present problem, the cut-off for the rejection region would be n Area=0. 025 8. 5 cut-off=9. 36

Practice n n n Here are the times my mom ran in the 10 trials. Test the null hypothesis that she can runs a 9: 00 mile on average. mom<-c(9. 5, 10, 8. 75, 9, 11. 2, 8. 65, 9. 6, 10. 2, 8. 8, 9. 8) What are the null and alternative hypotheses? What do you conclude? What would have happened if we had completed a two-sided test?

Comparison of one-sided and twosided tests Two-sided p-value is always twice one-sided pvalue. n Two-sided test is more conservative because the rejection region is split between the high and low side. For the one-sided test, the rejection region is only on the side of interest n Two-sided test most common in literature even though usually people know the direction of effect they are interested in detecting. n Picture n

Wait a minute Up to now, assumed we know the population variance (is this a good assumption? ) n How could we estimate the population variance? n – Sample variance!!! – Is the sample variance exactly equal to population variance? – How can we account for the additional uncertainty? n Now, we need to do a little math

t-distribution Assume Xi are iid normal n Normal distribution n n Chi-square distribution (Proof of this is given in Casella and Berger and in Inference I) n t-distribution- ratio of Normal (U) and chisquare (V)

t-distribution n Heavier tails than normal distribution – Accounts for additional variability – Tails heavier with fewer degrees of freedom (dof) n As dof goes to infinity, t dist normal dist n Can use t-dist test statistic just as the previous n Remember assumption of underlying normal

Example We can use a t-test to test the second null hypothesis about our pregnant teens, namely that the time from the first visit to the second visit is the same as in the general population n First, we need to ensure that the underlying distribution is approximately normal n

Steps for hypothesis testing Null: m=4. 3 weeks, Alternative: m != 4. 3 weeks 2) One sample hypothesis t-test, alpha=0. 05 1) 3) p-value = 0. 0017 5) Reject null 6) Conclusion: There is a difference in the amount of time from the first visit to the second visit. The time is longer for the pregnant teens. 4)

One sample t-test in R n To complete a t-test in R, use > t. test(second, mu=4. 8) One Sample t-test data: second t = 3. 4035, df = 34, p-value = 0. 00172 alternative hypothesis: true mean is not equal to 4. 8 95 percent confidence interval: 5. 271960 6. 670897 sample estimates: mean of x 5. 971429

Practice n Using the class data set, test the following hypotheses: – The average of an incoming student to the biostat program is 25. Is the mean age of this year’s class significantly different? Is there anything we need to consider in this analysis? – The average height of an incoming student is 71 inches. Is the mean height of this year’s class significantly shorter?

More practice n The TV watching habits of my seventh grade classes are shown in the dataset TV. dat from the course website. The gender and age of the students is given as well. How did my students TV watching habits compare to the national average for 7 th graders of 4 hours/day? Use an alpha level of 0. 01.