Determining Sample Size 1 Statistical Significance What factors

Determining Sample Size 1

Statistical Significance What factors influence the probability of a statistical significance? 2

Statistical Significance What factors influence the probability of a statistical significance? ◦ ◦ Alpha Sample Size Amount of variability in sample Magnitude of differences between groups/categories/intervals 3

Determining Sample size 4

Determining Sample size 5

Where to get s? If we don’t have population data, how do we know or even estimate s? ◦ One solution take a small sample ◦ Not always practical 6

Sample Size for Proportion 7

Example 8

POWER 9

IS MY COIN FAKE? How many flips before you are confident coin is fake 10

IS MY COIN FAKE? How many flips before you are confident coin is fake? Number of heads Probability 1 0. 5 2 0. 25 3 0. 125 4 0. 0625 5 0. 03125 6 0. 015625 7 0. 007813 8 0. 003906 9 0. 001953 10 0. 000977 11

Relationship between Power and hypothesis testing Accept Null Hypothesis Reject Null Hypothesis is true Correct decision Type I error( alpha typically set to 5%) Null Hypothesis is False Type II error Correct decision: Probability of making this decision correctly is defined as Power Probability of making this correct inference 12

Requirements to estimate Power Type of test (e. g. two-sample independent t-test, one tail) Alpha Effect size of interest How much accuracy is desirable Sample size Standard deviation of sample 13

Requirements to estimate Power Type of test (e. g. two-sample independent t-test, one tail) ◦ Given 14

Requirements to estimate Power Alpha ◦ Prefer to avoid Type I error-reject null hypothesis although null hypothesis is true (lower alpha (. 01) ◦ Prefer to avoid Type II error –accept null hypothesis although null hypothesis is false (higher alpha (. 05) 15

Requirements to estimate Power Effect size of interest ◦ Determined by theory or intuition • ◦ Are men heavier than women? What is an “important” difference? • Two kilograms? • Twenty kilograms? 16

Requirements to estimate Power 17

Requirements to estimate Power 18

Stata Examples Class data ◦ Are men heavier than women? 10. 2 in text book Captain Beaver is warned by Colonel Verleaf that if the mean efficiency rating for the 150 platoons under Verleaf’s command falls below 80, Captain Beaver will be transferred to Minot Air Base (A base in the middle of nowhere). Beaver takes a sample of 20 platoos and finds the following: mean = 85; s=13. 5 ◦ ◦ Null hypothesis µ = 80 Alternative hypothesis µ = 85 sd = 13. 5 n = 20 19

Problem 10. 2 20

Stata Example Power for Proportion 12. 10 in Book VISTA manager William suspects 50% of his volunteers are over 65 years old. A survey of 16 volunteers reveals 44% that are over age 65. How much power does he have? 21

Problem 12. 10 22

Sample size and power 23

Effect Size and Power 24

Are incomes higher in Mixed Income Developments NYSHCR survey of tenants 0=Not mixed income, 1 = mixed income 25

Are incomes higher in Mixed Income Developments 26

Are incomes higher in Mixed Income Developments 27