Effect Size Power Analysis 0246 511 Statistics for

Effect Size & Power Analysis 0246 -511 Statistics for Graduate Study I 1

Learning Objectives § Understand common effect size measures § Which to use § When to use different types § Learn to employ effect sizes to plan study § A priori power analysis § Two groups § ANOVA § Observed power 2

Effect Size § Two Broad Types (in ANOVA) § Standardized mean differences § Proportion of variance accounted for § § § Compared to statistical significance APA Guidelines Use in Meta-Analysis 3

Effect Size: Comparing Two Means Standardized Mean Differences Population Value Estimated from sample Proportion of Variance Accounted for 4

Concepts (Two-Groups) § § § § Power Type I & Type II Error Effect Size γ (gamma) δ (delta) Estimating Power Estimating Needed Sample Size Observed Power 5

Review § § Power Type I Error Type II Error Factors that affect power § § 6

Independent Samples t-test: Things we need to estimate power § Estimate Effect Size § Estimate Gamma § Estimate Delta § Estimate Power § Working backwards to get N 7

An example Fire Safety Instruction § Two methods of teaching § Pilot Study § Need to estimate sample size needs to get funding § Power of. 80 at alpha. 05 and. 01 Score 1 2 3 X 1 77 67 60 X 2 77 80 81 4 5 6 7 8 9 10 79 65 71 68 77 69 80 82 60 70 74 74 77 83 8

We will need to § Estimate of Effect Size § Gamma § Estimate Delta § Estimate (observed) Power § Work backwards to get n 9

Results from Pilot Study Observed Power (. 05) ≈ 0. 32 Observed Power (. 01) ≈ 0. 14 Note: strictly speaking, all we need to estimate n for power of. 80 is gamma, we compute observed power for illustration and because it is sometimes helpful. 10

Obtaining estimate of n Note 1: these are not obtained delta estimates, but based on what is needed to obtain desired power level Note 2: these are “n” values not “N” values, thus need this many per condition. 11

Unequal Sample Sizes For unequal sample sizes we often use something called a harmonic mean. It is calculated as follows: If our group sizes had been 14 and 9: 12

Effect Sizes: More than two means Population Based Measure Estimated from Sample Another sample measure Also called R 2 13

A brief digression ω2 can be derived from the F statistic as well: You will care shortly. 14

Cohen’s Effect Size Estimate § Cohen defines a population ANOVA ES estimate as: § Stevens provides an estimate of this as: § Keppel & Wickens provide another estimate: 15

Extend previous example 16

Use power chart to obtain a noncentrality paramter: Φ 17

Using power charts to get sample size What is a noncentrality parameter? From examining a power chart, Φ≈1. 8 Now we substitute our obtained Φ value into the following equation: Rounding up to 51, this implies 3(51 -1)=150 for dfdenom. We need to iterate again. Using the infinity line, we get… 18

Strategy for assessing power § § § § Determine form of experiment/research Decide on hypothesis test Carefully consider which effect(s) is (are) important to detect Select desired power (& alpha) Determine likely effect size (or range of effect sizes) Conduct power analysis Consider whether study is feasible as currently designed 19

In-Class Example § Students’ (MA Thesis) involves trying to find a hypothesized effect. § Wish to detect effect it if it is at least ω2=. 03 § df numerator will be 1 § How many subjects will she have to recruit? 20

Software Options § SPSS (observed power) § G*Power (consistent with Dr. Jackson’s 2 nd principle of economics) § Web-based power calculators § Others 21