Hypothesis Testing Quantitative Methods in HPELS 6210 Agenda

  • Slides: 58
Download presentation
Hypothesis Testing Quantitative Methods in HPELS 6210

Hypothesis Testing Quantitative Methods in HPELS 6210

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One vs. Two Tailed Tests n Effect Size and Power n Instat n Example n

Introduction Hypothesis Testing n Recall: ¨ Inferential Statistics: Calculation of sample statistic to make

Introduction Hypothesis Testing n Recall: ¨ Inferential Statistics: Calculation of sample statistic to make predictions about population parameter ¨ Two potential problems with samples: n Sampling error n Variation between samples ¨ Infinite # of samples predictable pattern sampling distribution n Normal n µ = µM n M = /√n

Introduction Hypothesis Testing n n n Common statistical procedure Allows for comparison of means

Introduction Hypothesis Testing n n n Common statistical procedure Allows for comparison of means General process: State hypotheses 2. Set criteria for decision making 3. Collect data calculate statistic 4. Make decision 1.

Introduction Hypothesis Testing n Remainder of presentation will use following concepts to perform a

Introduction Hypothesis Testing n Remainder of presentation will use following concepts to perform a hypothesis test: Z-score ¨ Probability ¨ Sampling distribution ¨

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One vs. Two Tailed Tests n Effect Size and Power n Instat n Example n

General Process of HT n n Step 1: State hypotheses Step 2: Set criteria

General Process of HT n n Step 1: State hypotheses Step 2: Set criteria for decision making Step 3: Collect data and calculate statistic Step 4: Make decision

Step 1: State Hypotheses n Two types of hypotheses: Null Hypothesis (H 0): 2.

Step 1: State Hypotheses n Two types of hypotheses: Null Hypothesis (H 0): 2. Alternative Hypothesis (H 1): 1. § § n n Directional Non-directional Only one can be true Example 8. 1, p 223

Assume the following about 2 -year olds: µ = 26 =4 M = /√n

Assume the following about 2 -year olds: µ = 26 =4 M = /√n = 16 Researchers want to know if extra handling/stimulation of babies will result in increased body weight once the baby reaches 2 years of age

Null Hypothesis: Alternative Hypothesis: H 0: Sample mean = 26 H 1: Sample mean

Null Hypothesis: Alternative Hypothesis: H 0: Sample mean = 26 H 1: Sample mean ≠ 26

Reality: Only ONE sample will be chose Assume that this distribution is the “TRUE”

Reality: Only ONE sample will be chose Assume that this distribution is the “TRUE” representation of the population µM Recall: If an INFINITE number of samples are taken, the SAMPLING DISTRIBUTION will be NORMAL with µ = µM and will be identical to the population distribution What is the probability of choosing a sample with a mean (M) that is 1, 2, or 3 SD above or below the mean (µM)?

Inferential statistics is based on the assumption that our sample is PROBABLY representative of

Inferential statistics is based on the assumption that our sample is PROBABLY representative of the population µM It is much more PROBABLE that our sample mean (M) will fall closer to the mean of the means (µM) as well as the population mean (µ)

µM Our sample could be here, or here, but we assume that it is

µM Our sample could be here, or here, but we assume that it is here!

H 0: Sample mean = 26 If true (no effect): 1. ) It is

H 0: Sample mean = 26 If true (no effect): 1. ) It is PROBABLE that the sample mean (M) will fall in the middle 2. ) It is IMPROBABLE that the sample mean (M) will fall in the extreme edges H 1: Sample mean ≠ 26 If true (effect): 1. ) It is PROBABLE that the sample mean (M) will fall in the extreme edges 2. ) It is IMPROBABLE that the sample mean (M) will fall in the middle

µ = 26 M = 30 Assume that M = 30 lbs Accept or

µ = 26 M = 30 Assume that M = 30 lbs Accept or reject? (n = 16) H 0: Sample mean = 26 What criteria do you use to make the decision?

Step 2: Set Criteria for Decision n A sampling distribution can be divided into

Step 2: Set Criteria for Decision n A sampling distribution can be divided into two sections: ¨ Middle: Sample means likely to be obtained if H 0 is accepted ¨ Ends: Sample means not likely to be obtained if H 0 is rejected n Alpha (a) is the criteria that defines the boundaries of each section

Step 2: Set Criteria for Decision n Alpha: ¨ AKA n level of significance

Step 2: Set Criteria for Decision n Alpha: ¨ AKA n level of significance Ask this question: ¨ What degree of certainty do I need to reject the H 0? 90% certain: a = 0. 10 n 95% certain: a = 0. 05 n 99% certain: a = 0. 01 n

Step 2: Set Criteria for Decision n As level of certainty increases: ¨a decreases

Step 2: Set Criteria for Decision n As level of certainty increases: ¨a decreases ¨ Middle section gets larger ¨ Critical regions (edges) get smaller n Bottom line: A larger test statistic is needed to reject the H 0

Step 2: Set Criteria for Decision Directional vs. nondirectional alternative hypotheses n Directional: n

Step 2: Set Criteria for Decision Directional vs. nondirectional alternative hypotheses n Directional: n ¨ H 1: n Non-directional: ¨ H 1: n M > or < X M≠X Which is more difficult to reject H 0?

Step 2: Set Criteria for Decision n n Z-scores represent boundaries that divide sampling

Step 2: Set Criteria for Decision n n Z-scores represent boundaries that divide sampling distribution Non-directional: ¨a ¨a ¨a n = 0. 10 defined by Z = 1. 64 = 0. 05 defined by Z = 1. 96 = 0. 01 defined by Z = 2. 57 Directional: ¨a ¨a ¨a = 0. 10 defined by Z = 1. 28 = 0. 05 defined by Z = 1. 64 = 0. 01 defined by Z = 2. 33

Critical Z-Scores Non-Directional Hypotheses Z=1. 64 Z=1. 96 Z=2. 58 90% 95% 99%

Critical Z-Scores Non-Directional Hypotheses Z=1. 64 Z=1. 96 Z=2. 58 90% 95% 99%

Critical Z-Scores Directional Hypotheses Z=1. 28 Z=1. 64 Z=2. 34 90% 95% 99%

Critical Z-Scores Directional Hypotheses Z=1. 28 Z=1. 64 Z=2. 34 90% 95% 99%

Step 2: Set Criteria for Decision n Where should you set alpha? ¨ Exploratory

Step 2: Set Criteria for Decision n Where should you set alpha? ¨ Exploratory research 0. 10 ¨ Most common 0. 05 ¨ 0. 01 or lower?

Step 3: Collect Data/Calculate Statistic n Z = M - µ / M where:

Step 3: Collect Data/Calculate Statistic n Z = M - µ / M where: M = sample mean ¨ µ = value from the null hypothesis ¨ n ¨ H 0: sample = X M = /√n n Note: Population must be known otherwise the Z-score is an inappropriate statistic!!!!!

Step 3: Collect Data/Calculate Statistic n Example 8. 1 Continued

Step 3: Collect Data/Calculate Statistic n Example 8. 1 Continued

M = 30 Assume the following about 2 -year olds: µ = 26 Researchers

M = 30 Assume the following about 2 -year olds: µ = 26 Researchers want to know if extra handling/stimulation of babies will result in increased body weight once the baby reaches 2 years of age =4 M = /√n = 1 Z = M - µ / M n = 16 Z = 30 – 26 / 1 Z = 4 / 1 = 4. 0

Step 4: Make Decision n Process: 1. Draw a sketch with critical Z-score §

Step 4: Make Decision n Process: 1. Draw a sketch with critical Z-score § § Assume non-directional Alpha = 0. 05 Plot Z-score statistic on sketch 3. Make decision 2.

Step 1: Draw sketch Critical Z-score Z = 1. 96 µ = 26 M

Step 1: Draw sketch Critical Z-score Z = 1. 96 µ = 26 M = 30 Step 2: Plot Z-score Z = 4. 0 Step 3: Make Decision: Z = 4. 0 falls inside the critical region If H 0 is false, it is PROBABLE that the Z-score will fall in the critical region ACCEPT OR REJECT?

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One vs. Two Tailed Tests n Effect Size and Power n Instat n Example n

Errors in Hypothesis Testing n Recall Problems with samples: ¨ Sampling error ¨ Variability

Errors in Hypothesis Testing n Recall Problems with samples: ¨ Sampling error ¨ Variability of samples Inferential statistics use sample statistics to predict population parameters n There is ALWAYS chance for error n

Errors in Hypothesis Testing n 1. 2. There is potential for two kinds of

Errors in Hypothesis Testing n 1. 2. There is potential for two kinds of error: Type I error Type II error

Type I Error Rejection of a true H 0 n Recall alpha = certainty

Type I Error Rejection of a true H 0 n Recall alpha = certainty of rejecting H 0 n ¨ Example: n Alpha = 0. 05 n 95% certain of correctly rejecting the H 0 n Therefore 5% certain of incorrectly rejecting the H 0 Alpha maybe thought of as the “probability of making a Type I error n Consequences: n ¨ False report ¨ Waste of time/resources

Type II Error Acceptance of a false H 0 n Consequences: n ¨ Not

Type II Error Acceptance of a false H 0 n Consequences: n ¨ Not as serious as Type I error ¨ Researcher may repeat experiment if type II error is suspected

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One vs. Two Tailed Tests n Effect Size and Power n Instat n Example n

One vs. Two-Tailed Tests n One-Tailed (Directional) Tests: ¨ Specify an increase or decrease

One vs. Two-Tailed Tests n One-Tailed (Directional) Tests: ¨ Specify an increase or decrease in the alternative hypothesis ¨ Advantage: More powerful ¨ Disadvantage: Prior knowledge required

One vs. Two-Tailed Tests n Two-Tailed (Non-Directional) Tests: ¨ Do not specify an increase

One vs. Two-Tailed Tests n Two-Tailed (Non-Directional) Tests: ¨ Do not specify an increase or decrease in the alternative hypothesis ¨ Advantage: No prior knowledge required ¨ Disadvantage: Less powerful

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One vs. Two Tailed Tests n Effect Size and Power n Instat n Example n

Recall Step 4: Make a Decision Statistical Software p-value n The p-value is the

Recall Step 4: Make a Decision Statistical Software p-value n The p-value is the probability of a type I error n Recall alpha (a) n

Recall Step 4: Make a Decision n If the p-value > a accept the

Recall Step 4: Make a Decision n If the p-value > a accept the H 0 ¨Probability of type I error is too high ¨Researcher is not “comfortable” stating that differences are real and not due to chance n If the p-value < a reject the H 0 ¨Probability of type I error is low enough ¨Researcher is “comfortable” stating that differences are real and not due to chance

Statistical vs. Practical Significance n 1. 2. Distinction: Statistical significance: There is an acceptably

Statistical vs. Practical Significance n 1. 2. Distinction: Statistical significance: There is an acceptably low chance of a type I error Practical significance: The actual difference between the means are not trivial in their practical applications

Practically Significant? n n Knowledge and experience Examine effect size ¨ The n magnitude

Practically Significant? n n Knowledge and experience Examine effect size ¨ The n magnitude of the effect Examples of measures of effect size: ¨ Eta-squared ¨ Cohen’s d ¨ R 2 n (h 2) Interpretation of effect size: ¨ 0. 0 – 0. 2 = small effect ¨ 0. 21 – 0. 8 = moderate effect ¨ > 0. 8 = large effect n Examine power of test

Statistical Power Statistical power: The probability that you will correctly reject a false H

Statistical Power Statistical power: The probability that you will correctly reject a false H 0 n Power = 1 – b where n ¨b n = probability of type II error Example: Statistical power = 0. 80 therefore: ¨ 80% chance of correctly rejecting a false H 0 ¨ 20% of accepting a false H 0 (type II error)

Researcher Conclusion Accept H 0 Reality No real difference About exists Test Real difference

Researcher Conclusion Accept H 0 Reality No real difference About exists Test Real difference exists Reject H 0 Correct Type I Conclusion error Type II error Correct Conclusion

Statistical Power n 1. What influences power? Sample size: As n increases, power increases

Statistical Power n 1. What influences power? Sample size: As n increases, power increases - Under researcher’s control 2. Alpha: As a increases, b decreases therefore power increases - Under researcher’s control (to an extent) 3. Effect size: As ES increases, power increases - Not under researcher’s control

Statistical Power How much power is desirable? n General rule: Set b as 4*a

Statistical Power How much power is desirable? n General rule: Set b as 4*a n Example: n ¨a = 0. 05, therfore b = 4*0. 05 = 0. 20 ¨ Power = 1 – b = 1 – 0. 20 = 0. 80

Statistical Power n What if you don’t have enough power? ¨ More n subjects

Statistical Power n What if you don’t have enough power? ¨ More n subjects What if you can’t recruit more subjects and you want to prevent not having enough power? ¨ Estimate optimal sample size a priori ¨ See statistician with following information: n n n Alpha Desired power Knowledge about effect size what constitutes a small, moderate or large effect size relative to your dependent variable

Statistical Power n 1. 2. Examples: Novice athlete improves vertical jump height by 2

Statistical Power n 1. 2. Examples: Novice athlete improves vertical jump height by 2 inches after 8 weeks of training Elite athlete improves vertical jump height by 2 inches after 8 weeks of training

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One vs. Two Tailed Tests n Instat n Example n

Instat n Type data from sample into a column. ¨ Label column appropriately. n

Instat n Type data from sample into a column. ¨ Label column appropriately. n n Choose “Statistics” ¨ Choose “Simple Models” n n Choose “Manage” Choose “Column Properties” Choose “Name” Choose “Normal, One Sample” Layout Menu: § Choose “Single Data Column”

Instat n Data Column Menu: ¨ n Choose variable of interest. Parameter Menu Choose

Instat n Data Column Menu: ¨ n Choose variable of interest. Parameter Menu Choose “Mean, Known Variance (z-interval)” ¨ Enter known SD or variance value. ¨ n Confidence Level: 90% = alpha 0. 10 ¨ 95% = alpha 0. 05 ¨

Instat n Check “Significance Test” box: Check “Two-Sided” if using non-directional hypothesis. ¨ Enter

Instat n Check “Significance Test” box: Check “Two-Sided” if using non-directional hypothesis. ¨ Enter value from null hypothesis. ¨ n n n What population value are you basing your sample comparison? Click OK. Interpret the p-value!!!

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One

Agenda Introduction n Hypothesis Testing General Process n Errors in Hypothesis Testing n One vs. Two Tailed Tests n Instat n Example n

Example (p 246) n Researchers want to investigate the effect of prenatal alcohol on

Example (p 246) n Researchers want to investigate the effect of prenatal alcohol on birth weight in rats ¨ Independent variable? ¨ Dependent variable? n Assume: ¨ µ = 18 g ¨ = 4 ¨ n = 16 ¨ M = /√n ¨ M = 15 g = 4/4 = 1

Step 1: State hypotheses (directional or non-directional) H 0: µalcohol = 18 g H

Step 1: State hypotheses (directional or non-directional) H 0: µalcohol = 18 g H 1: µalcohol ≠ 18 g Step 2: Set criteria for decision making Alpha (a) = 0. 05 Step 3: Sample data and calculate statistic Z = M - µ / M Z = 15 – 18 / 1 = -3. 0

Step 4: Make decision Does Z-score fall inside or outside of the critical region?

Step 4: Make decision Does Z-score fall inside or outside of the critical region? Accept or reject? Statistical Software: p-value = 0. 02 Accept or reject? p-value = 0. 15 Accept or reject?

Homework n Problems: 3, 5, 6, 7, 8, 11, 21

Homework n Problems: 3, 5, 6, 7, 8, 11, 21