Statistical Terminology PSYA 4 TYPE 1 AND 2

Statistical Terminology PSYA 4

TYPE 1 AND 2 ERRORS • Type I Error (false positive) • In a hypothesis test, a type I error occurs when the null hypothesis is rejected when it is in fact true; that is, it is wrongly rejected. • In other words if we accept the alternative hypothesis when in fact the null was true. • E. g. in a clinical trial of a new drug, the null hypothesis might be that the new drug is no better, on average, than the current drug • A type I error would occur if we concluded that the two drugs produced different effects when in fact there was no difference between them.

Type II Error (false negative) • In a hypothesis test, a type II error occurs when the null hypothesis, is not rejected when it is in fact false. • it occurs if we accept the null hypothesis when in fact the alternative hypothesis was true • E. g. in a clinical trial of a new drug, null hypothesis might be that the new drug is no better, on average, than the current drug. • A type II error would occur if it was concluded that there is no difference between the two drugs on average, when in fact they produced different ones. • A type II error is frequently due to sample sizes being too small.

Significance level (p value) • The concept of statistical significance is central to statistics. • A result that is statistically significant is unlikely to have occurred by chance. • we can never be 100% sure that chance has not played a part. • However, when the likelihood of obtained results having occurred by chance is slim, researchers will reject the null hypothesis and accept the alternative hypothesis.

• The level of significance is a value used as a criterion for deciding whether the results occurred due to the IV or chance. • Significance levels may be written as percentages or decimals. • 5% level of significance means that there is a 5% probability that the results occurred by chance and a 95% chance that the results occurred due to the IV. • This can also be written as p 0. 05 where p = the probability of the results being due to chance.

• To be more precise the p value should be written as p 0. 05 meaning that the probability that the results occurred due to chance are 5% or less.

Which significance level is used? • Popular levels of significance are 5% (0. 05), 1% (0. 01) and 0. 1% (0. 001). • If a statistical test gives a p-value lower than the significance level, the null hypothesis is rejected. • Such results are referred to as 'statistically significant'.

Example • if someone argues that "there's only one chance in a thousand this could have happened by coincidence, " a 0. 001 level of statistical significance is being implied. • The lower the significance level, the stronger the evidence required. • Choosing level of significance is an arbitrary task, but usually a level of 5% is chosen. • This means that the likelihood of obtaining these results by chance is less than or equal to 1 in 20.

Link to type 1 & 2 errors • type I & II errors are inversely related; the smaller the risk of one, the higher the risk of other • The level of the p value needs to be chosen to balance the risk of making type 1 & 2 errors. • At the p=0. 05 level there is a 5% chance of making a type 1 error. • If a level of 1% is chosen there is only a 1% chance of making a type 1 error but a greater chance of making a type 2 error than at the 5% level