TYPE I ERROR AND TYPE II ERROR hypothetical

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 5% level 2. 5% b 20 -1. 96 sd b 20 -sd b 20+sd b 20+1. 96 sd b 2 In the previous sequence a Type I error was defined to be the rejection of a null hypothesis when it happens to be true. 1

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 5% level 2. 5% b 20 -1. 96 sd b 20 -sd b 20+sd b 20+1. 96 sd b 2 In hypothesis testing there is also a possibility of failing to reject the null hypothesis when it is in fact false. This is known as a Type II error. 2

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 5% level 2. 5% b 20 -1. 96 sd b 20 -sd b 20+sd b 20+1. 96 sd b 2 This sequence will demonstrate that there is a trade-off between the risk of making a Type I error and the risk of making a Type II error. 3

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 5% level 2. 5% b 20 -1. 96 sd b 20 -sd b 20+sd b 20+1. 96 sd b 2 The diagram show the acceptance region and the rejection regions for a 5% significance test. The risk of making a Type I error, if the null hypothesis happens to be true, is 5%. 4

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level 5% level 0. 5% b 20 -2. 58 sd 0. 5% b 20 -sd b 20+sd b 20+2. 58 sd b 2 Using a 1% significance test, instead of a 5% test, reduces the risk of making a Type I error to 1%, if the null hypothesis is true. 5

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level 5% level 0. 5% b 20 -2. 58 sd 0. 5% b 20 -sd b 20+sd b 20+2. 58 sd b 2 We will consider the implications of the choice of significance test for the case where the null hypothesis happens to be false. 6

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level 5% level 0. 5% b 20 -2. 58 sd 0. 5% b 20 -sd b 20+sd b 20+2. 58 sd b 2 The diagram above explains how the test decisions are made, but it does not give the actual distribution of b 2. (For that reason the curve has been drawn with a dashed line. ) 7

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 Suppose that H 1: b 2 = b 21 is in fact true and the distribution of b 2 is therefore governed by the right-hand curve. 8

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 If we obtain some data and run a regression, the estimate b 2 might be as shown. In this case we would make the right decision and reject H 0, no matter which test we used. 9

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 Here is another estimate. Again, we would make the right decision and reject the null hypothesis, no matter whether we use the 5% test or the 1% test. 10

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 In the case shown, we would make a Type II error and fail to reject the null hypothesis, using either significance level. 11

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 But in the case of this estimate, we would make the right decision if we used a 5% test but we would make a Type II error if we used a 1% test. 12

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 The probability of making a Type II error if we use a 1% test is given by the probability of b 2 lying within the 1% acceptance region, the interval between the red vertical dotted lines. 13

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 Given that H 1 is true, the probability of b 2 lying in the acceptance region is that area under the distribution for H 1 in the diagram - the pink shaded area in the diagram. 14

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 If instead we use a 5% significance test, the probability of making a Type II error if H 1 is true is given by the area under the distribution for H 1 in the acceptance region for the 5% test. 15

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 This is the gray shaded area in the diagram. In this particular case, using a 5% test instead of a 1% test would approximately halve the risk of making a Type II error. 16

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 The problem, of course, is that you never know whether H 0 is true of false. If you did, why would you be performing a test? 17

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 If H 0 happens to be true, using a 1% test instead of a 5% test greatly reduces the risk of making a Type I error (you cannot make a Type II error). 18

TYPE I ERROR AND TYPE II ERROR hypothetical distribution under acceptance region for b 2 1% level actual distribution under 5% level 0. 5% b 20 b 21 -2 sd b 21 -sd b 21+sd b 21+2 sd b 2 However, if H 0 is false, using a 1% test instead of a 5% test increases the risk of making a Type II error (you cannot make a Type I error). 19