Power and Multiple Regression 1 Relationship between Power

Power and Multiple Regression 1

Relationship between Power and hypothesis testing Accept Null Hypothesis is true Correct decision Null Hypothesis is False Reject Null Hypothesis Type I error( alpha typically set to 5%) Type II error (aka Beta) Correct decision: Probability of making this decision correctly is defined as Power 2

Why is Power important? Insure sample is large enough to detect effect of interest 3

Requirements to estimate Power Alpha Effect size of interest Sample size 4

Requirements to estimate Power Effect size of interest Determined by theory or intuition Are men heavier than women? What is an “important” difference? Two kilograms? Twenty kilograms? 5

Requirements to estimate Power Alpha § Risk of committing Type I error (rejecting null hypothesis when it is true) vs Type II error (accepting null hypothesis when false) 6

Power in Stata § Powerreg command: gives power estimates for changes in R 2 § Example: Predicting student’s weight based on height § Requirements: § Alpha: We will use conventional. 05 § Effect size: What impact do we expect the use height to have on R 2? § § This is based on prior research, experience, intuition or theory § Let’s use. 20 or a 20% increase in r-square Sample size: 22 students in class 7

Power in Stata 8

Power in Stata § When choosing effect size of interest you are choosing the change in R-square of interest § In most instances you will not have strong a priori reasons for a specific effect size or change in R-square § For the assignment 5% is reasonable 9

Power for Logistic Regression § Use powerlog in Stata § Gives power estimates for changes in predicted probabilities § Requirements: § Alpha: § Effect size: § Sample size: 10

Power for Logistic Regression § Use powerlog in Stata § Gives power estimates for changes in predicted probabilities § Requirements: § Alpha: Conventional. 05 § Effect size: § § Predicted probability at mean of independent variable (P 1) § Predicted probability at mean + 1 SD of independent variable (P 2) Sample size: To be determined by program 11

Power for Logistic Regression § Example: Power for baseball team making playoffs based on ERA § Gives power estimates for changes in predicted probabilities § Requirements: § Alpha: Conventional. 05 § Effect size: § § Predicted probability at mean of ERA =. 25 (P 1) § Predicted probability at mean + 1 SD of independent variable =. 13 (P 2) Sample size: To be determined by program 12

Obtaining predicted probabilities for independent variables If you have the data Use summarize command to obtain standard deviation Use margins command to produce predicted probabilities E. g. margins, at independent_variable=(1, 2, 3…)) If you don’t have data Use published research Make educated guess

Power for Logistic Regression 14

Power for Logistic Regression using. 025 to assume two tailed test 15

Power for Logistic Regression using different predicted probabilities 16

For Assignment § § If using OLS regression § Estimate power for bivariate model (you can additional variables for the rest of assignment) § Estimate power for a r-square and an increase in r-square § Use an r-square consistent with prior research, or § Use a change of five percentage points If using Logistic Regression § Use power analysis for bivariate model (you can additional variables for the rest of assignment) § Use two predicted probabilities to estimate power § Determine if sample is sufficient for desired power (e. g. . 8) 17