Power and Error What is it Hypothesis Testing

Hypothesis Testing • Remember that tests of significance are to assess the evidence the

Types of wrong decisions • TYPE 1 ERROR: if we reject the null hypothesis

Type I error • Is set by the statistician! • Remember: – P-value: probability

Type II Error • This is a much harder calculation! • This assumes that

POWER • The probability at a fixed significance level that we will reject the

fail to reject Ho type II error reject Ho power

fail to reject Ho What if we increased alpha? type II error power and

fail to reject Ho type II error reject Ho What if we increased sample

fail to reject Ho What if the alternative (truth? ) were further from H

So it’s a balancing act. What is most important in the context of the

To raise the power of a test (that is, to increase the probability of

Slides: 12

Download presentation

Power and Error What is it?

Hypothesis Testing • Remember that tests of significance are to assess the evidence the null hypothesis • When statisticians make decisions based off the strength of evidence against a null hypothesis they also must also worry about the possiblity of making a wrong decision.

Types of wrong decisions • TYPE 1 ERROR: if we reject the null hypothesis when it is actually true. • TYPE 2 ERROR: if we fail to reject the null when if fact the Alternative is true.

Type I error • Is set by the statistician! • Remember: – P-value: probability of a null hypothesis being true given then sample data. – α= probability that the test will reject the null hypothesis Therefore: α is the probability of type I error!!

Type II Error • This is a much harder calculation! • This assumes that the alternative is true but we failed to reject the null. • To calculate this error we need to know what the alternative parameter actually is… this is rarely known! • High type II error implies that the test is not sensitive enough to detect the alternative.

POWER • The probability at a fixed significance level that we will reject the null when the particular alternative value is true! • POWER= 1 -type II error. • P-values describes the probability of a situation given the null is true. • Power describes the probability given a particular alternative value is true!

fail to reject Ho type II error reject Ho power

fail to reject Ho What if we increased alpha? type II error power and the power increases!

fail to reject Ho type II error reject Ho What if we increased sample size? Well, the standard deviation would get smaller. power What happened to power? To a type II error?

fail to reject Ho What if the alternative (truth? ) were further from H o? type II error is near 0 power is near 1

So it’s a balancing act. What is most important in the context of the problem? Which error would be most costly or dangerous?

To raise the power of a test (that is, to increase the probability of correctly rejecting Ho if it is NOT true) 1. increase sample size 2. decrease variation 3. move alternative further from Ho 4. increase alpha (prob. of type I error)