Chapter 9 Significance Testing Section 1 The Basics

Chapter 9: Significance Testing Section 1: The Basics

Definitions �Significance Testing: ◦ A formal procedure for comparing observed data with a hypothesis whose truth value we want to assess. �Hypothesis: ◦ A statement about a population parameter, like µ or p.

Let’s think… �Several cities have begun to monitor and examine paramedic response times. In Hurtville, the mean response time to all accidents involving life threatening accidents was µ=6. 7 minutes. The city manager encourages them to do better each year. The following year the city manager selects a SRS of 400 life threatening calls. The average response time for this sample was 6. 48 minutes. �Do these data points provide good evidence that the response times have decreased since last year?

Let’s think… �What would we want to test? ◦ Normality? �Standard deviation would be… ◦ For or Against claim? �Against the claim!

Types of Hypotheses �Null Hypothesis, H 0: ◦ The statement we are testing ◦ Statement of “no effect, ” “no difference, ” or no change in historical values �Alternative Hypothesis, H 1 or Ha: ◦ The claim about the population we are trying to find evidence for ◦ Suggests that something has changed or different than expected

Looking at the paramedics… �If we look back at the response times for our paramedics, what would be our null hypothesis and alternative hypothesis? ◦ ◦ ◦ This is a one-sided alternative since we are only concerned with deviations in one direction.

Example 1: � Larry’s car averages 26 mpg on the highway. He switches to a brand of new motor oil that is advertised to increase gas mileage. After driving 3000 highway miles with the new oil, he wants to determine if the average gas mileage has increased. � What are the appropriate null and alternative hypotheses? Be sure to use the appropriate parameters and define them. ◦ µ = mean gas mileage for Larry’s car on the highway

Example 2: �A May 2005 Gallup Poll report on a national survey of 1208 teens revealed 72% of teens said they rarely or never argue with their friends. You wonder whether this would be true in your school so you conduct your own survey of random sample of students in your school. � What are the appropriate null and alternative hypotheses? Be sure to use the appropriate parameters and define them. ◦ p = the proportion of teens in your school who rarely argue with their friends

Test Statistics �Compares the value of the parameter as stated in the null hypothesis with an estimate of the parameter from the sample data. �Values of the estimate far from the parameter value in the direction specified by the alternate hypothesis give evidence against the null.

Test Statistics �To assess how far from the parameter, we standardize the estimate:

Looking at the paramedics… �Given the paramedic example where the average for the sample of 400 calls was 6. 48, find the test statistic we would want to use to test our hypotheses.

P-values �The probability, assuming H 0 is true, that the observed outcome would take a value as extreme as or more extreme than actually observed. �The smaller the p-value, the stronger the evidence is against H 0 provided by the data (ie observed result is unlikely to occur when H 0 is true). �Larger p-values fail to give evidence against H 0.

Looking at the paramedics… �What is the p-value associated with testing our null hypothesis? ◦ Using our test statistic: ◦ The p-value would be: �So…are we for or against our H 0 ? ◦ Small p-value provides strong evidence against H 0

Example 3: � Does the job satisfaction of assembly workers differ when their work is machine-paced rather than self-paced? Assume the standard deviation of this normal distribution is 60. If there is no difference in job satisfaction, then the mean is 0. One study chose 18 subjects at random from a group of people who assembled electronics. These workers gave a mean of 17 (that is these workers preferred the self-paced environment). � Hypotheses: � Test Statistic: � P-Value:

Statistical Significance �If the p-value is as small as or smaller than the alpha, we say that the data as statistically significant at level �At a 95% confidence level, �Note: Significant in a statistical sense, does not mean “important. ” It just means simply not likely to happen just by chance.

Statistical Significance �If the P =. 03 under a 95% confidence level, then �So is the result statistically significant? ◦ Yes �What ◦ No about for

Interpretations �So…we find the p-value… and we know if it’s statistically significant…now what… �We want to reject H 0 or fail to reject H 0 ◦ If p-value > then we fail to reject H 0 ◦ If p-value < then we reject H 0

Steps for Significance Tests 1. Hypothesis: Identify the population and parameter you want to draw conclusion about. State the hypothesis 2. Conditions: Choose the appropriate inference procedure. Verify conditions. 3. Calculations: Find test statistic and p-value 4. Interpret: Decide statistical significance and write conclusion.

Alternative Hypothesis, Ha � � �

Fail to reject H 0 : ◦ We cannot find evidence to suggest Ha is true. ◦ Thus for 2 tail, we cannot find convincing evidence that the mean differs ◦ Only means the data are consistent with H 0, not that we have evidence that H 0 is true. ◦ Want to make statement that “since the obtained p-value is so large, we don’t have strong evidence against H 0 ”

Errors in Hypothesis Tests

When you perform a hypothesis test you make a decision: reject H 0 or fail to reject H 0 When you make one of these decisions, there is a possibility that you could be wrong! That you made an error!

There are two decisions that we make; reject or fail to reject. Each could possibly be a wrong decision; therefore, there are two types of errors.

Type I error �When you reject the null hypothesis that is really true �Denoted by a ◦ Is the level of significance of the test a m 0

Type II error �When you fail to reject the null hypothesis when it is false �Denoted by b Fail to reject Reject a b Type I – reject a The true x-bar. His 0 really The x-bar is really part of the Ha curve, 0 curve, but we mistake it as being part of the H 0 Type part II – of the Ha curve fail to curve reject a false H 0

Let’s start exploring �Follow instructions on hand out and visit the appropriate website. �Justice �Ideas System behind statistical Power