HYPOTHESIS TESTING FOR DIFFERENCES BETWEEN MEANS AND BETWEEN

HYPOTHESIS TESTING FOR DIFFERENCES BETWEEN MEANS AND BETWEEN PROPORTIONS

ASSUMPTIONS Selecting an appropriate test for analysing hypotheses of difference depends on a number of important assumptions particularly relating to parametric and non-parametric assumptions

ASSUMPTIONS : PARAMETRIC VERSUS NON-PARAMETRIC TESTS • Parametric tests – Tests based on assumptions about population distributions and parameters The assumptions for parametric tests – – – interval or ratio level data (SPSS Scale data) normal distribution or closely so homogeneity of variance - the variance (standard deviation squared) should be similar in each group samples randomly drawn from the population • Non-Parametric Tests – Tests that make no assumptions about population parameters or distributions

Designs Hypotheses of differences can involve different research designs • Independent Groups or Between Groups Design – Two independent samples are selected randomly and compared often in a control versus experimental contrast. • Repeated Measures Design – One sample is measured in both conditions – a before and after approach. • Paired or Matched Samples Design – Individuals, units or observations are matched as pairs on one or more variables and allocated at random one to each sample. Different forms of the t test and ANOVA exist for these designs

t -TEST t Tests are parametric tests and assume • • normal distribution or approximately so, random selection of sample elements, homogeneity of variance or approximately so and scale data measurement.

Types of t Test Three main types of t test exist: 1. The one sample t test 2. The independent group t test between two separate random samples 3. The repeated (or paired) measures t test between two testings of the sample or between two paired samples

THE ONE SAMPLE t TEST : TESTING HYPOTHESES FOR SINGLE SAMPLES t = sample mean – population mean estimated standard error of mean (derived from sample) (notice similarity to Z. t = Z when sample size is large) – tests the null hypothesis that the mean of a particular sample differs from the mean of the population only by chance

One sample t test and Z test Confidence intervals for a nominated level of probability provide a range of values within which the population mean is likely to lie. Using the same sort of logic, a different approach is a one sample test. In a one sample test, you nominate a known or possible population mean and you conduct the test to determine whether it is likely (with a nominated level of probability) that the mean value you have obtained from your sample could have come from the population with the nominated population mean.

Example from SPSS 1. Enter data as a single variable (here as “runs”) 2. Analyze > Compare Means > One-Sample t-test (normally we use more than 5 scores)

Example using SPSS 3. 4. 5. 6. 7. Enter the variable (in this case runs) into the Test Variables box. Enter the hypothesized population mean into the TEST VALUE box. Click options Enter 95% as the Confidence Interval. Click CONTINUE and OK

Example using SPSS 8. The calculated probability (Sig. (2 -tailed)) is. 257 which is greater than. 05 – So the result is not significant. t = 1. 319, p =. 257 so your mean of 7. 4 could have come from a distribution with a population mean of 5. 7

Comparing Means With t Tests More frequently we want to compare two sample means. There are two types of such comparisons. 1. Where we have two means which are calculated from two different samples (the populations are independent). e. g. if we wanted to compare attitudes of males with those of females towards some issue and determine whether one gender was more positive than the other a two-sample ttest may be used here. 2. Where we have two means which are calculated from the sample when we have measured the variable at two different times. E. g. Change in attitude of the whole group over time, a Repeated Measures (Paired Difference) ttest may be used.

Independent Groups t Test. Between Groups Design • The Independent Groups t test is used to determine whether two random samples are likely to come from the same population. • If there is a statistically significant difference between them then the null hypothesis that their sample means are simply two chance variations around a population mean is rejected. • Often used in the experimental versus control group set-up – the Between Subjects design.

ASSUMPTIONS FOR INDEPENDENT GROUPS t TEST – measurements are on interval scale – subjects are randomly selected from a defined population – the variances of the scores for the two samples or occasions should be approximately equal (homogeneity) – the population from which the samples have been drawn is normally distributed.

STATISTICAL TESTS FOR TWO INDEPENDENT GROUPS (BETWEEN-SUBJECTS DESIGN) - COMPARING TWO SAMPLES This test determines whether two sample group means differ significantly or not. The bigger the difference between the 2 sample means, and the smaller the variation within each group the more likely the population means are different.

The Standard Error of the Difference • We evaluate whether the sample means differ significantly using the t statistic: t = obtained difference between means standard error of the difference • The standard error of the difference is the standard deviation of the distribution of differences between every possible pairings of sample means when each pair is formed from one sample mean taken from each population

Example of Independent t test • Suppose we want to compare two parallel classes of students studying Tax Law. At the beginning of the year, students were randomly assigned to each class. • Each class is taught using a different method; one by usual face-to face teaching ; the other by individual study using a computerized module presentation. • After 6 months, we test whether performance (on a test with a maximum value of 20) varies between the two classes (two-tailed non-directional test). • For alpha =. 05

SPSS Example of Independent t test 1. 2. Enter your dependent variable (Tax Law test score) as a single variable. (usually we use at least 20 scores) Enter group membership (here “class”) as a second variable.

SPSS Example of Independent t test 3. 4. 5. Analyze > Compare Means > Independent-Samples t-test Enter your dependent variable (“score”) in the TEST VARIABLES box, and your grouping variable (“class”) in the GROUPING VARIABLE box. Click DEFINE GROUPS

SPSS Example of Independent t test 5. 6. 7. Enter the codes you used to define your two groups (here “ 1” and “ 2”). Click Continue Click OK

SPSS Example of Independent t test 8. Interpret: The mean for Class 1 = 7. 40, for Class 2 = 11. 80, d. f. = 8, t 8 = -2. 63. The calculated probability for such a mean difference =. 030, which is less than the alpha of. 05, and so is significant. 9. So we conclude that the performance of the two classes is significantly different, t 8 = -2. 63, p <. 05, with class two doing significantly Mean difference better. being tested Levine’s test suggests homogeneity of variance assumption OK p <. 05

DEGREES OF FREEDOM • Abbreviated to df • The number of values free to vary in a set of values • Used to evaluate the obtained statistical value rather then N • df usually = N – 1 per sample (group)

Effect Size for the t Test for Independent Groups • Effect Size – the difference between the population means divided by the standard deviation of the population of individuals – d = t N 1 + N 2 N 1 N 2

MANN-WHITNEY U TEST • Used when testing for differences between two independent groups when the assumptions for the parametric t test cannot be met: – data may be ordinal – variances are quite dissimilar – samples are small, distribution not close to normal. • Ranks scores from the two samples into one set of ranks - then determines whethere is a systematic clustering into two groups paralleling the samples • Does not require equal numbers in the two conditions • Less likely to show a significant difference between groups if one really exists

STATISTICAL TESTS FOR THE REPEATED MEASURES AND MATCHED PAIRS DESIGN (WITHIN-SUBJECTS DESIGN) • Used : – when the same subjects are tested under two different conditions, e. g. undertaking a task with two different sets of instructions – in a before-after type of study where changes after an intervention are compared with the performance level before – when subjects in two groups are paired by selecting individuals who are as similar as possible with respect to other external variables which may all influence the outcome of the research. Subjects might be matched for learning ability, sex, age and IQ.

Repeated Measures or Matched Pairs Design (Parametric test) • Assumptions of the Test – measurements are on interval scale – subjects are randomly selected from a defined population – the variances of the scores for the two samples or occasions should be approximately equal – the population from which the samples have been drawn is normally distributed.

Repeated Measures or Paired Samples SPSS Example • • Suppose we want to compare the effects of a training course on performance at two points in time. (a) prior to start of course, and (b) at end of course. Use a two-tailed test at alpha =. 05 Enter the data for performance at the beginning of the year as a variable (“before”). Enter the data for performance at the end of the year as a variable (“after”). (We would normally use at least 20 pairs of scores)

Repeated Measures or Paired Samples SPSS example. 3. 4. 5. Analyze > Compare Means > Paired-Samples t-test Select both variables (“before” and “after” and enter into PAIRED VARIABLES box) Click Options.

Repeated Measures or Paired Samples SPSS example. 6. 7. 8. Select 95% as your confidence interval. Click Continue. Click OK

Repeated Measures or Paired Samples SPSS example. 9. The mean for Before = 8. 40, After = 11. 80, d. f. = 4, t 4 = -13. 880, the calculated probability is. 000, which is less than the alpha level of. 05 – and so is significant. 10. So we conclude that the performance of the class has significantly changed from before-training to aftertraining, t 4 = -13. 88, p <. 001. We compare this mean difference p is highly significant

THE WILCOXON SIGNED RANKS TEST (NON-PARAMETRIC) • used instead of a related groups or paired t test if one or more of the following apply: – the differences between treatments can only be ranked in size, – the data are quite skewed, – there is clearly a difference in the variance of the groups • compares observations across two occasions or conditions in a repeated measures or matched pairs context to determine whethere are significant differences between the observations from the two sets of data from the same group.

Choosing the Correct Test. Independent Samples Tests (Between Groups Design) Level of measurement Two independent groups More than two independent groups Scale (interval and ratio) Independent samples t test One way ANOVA Ordinal Nominal Mann-Whitney Chi Square Kruskal-Wallis Chi Square Related Samples tests (Repeated and Paired Measures Design) Scale (interval and ratio) Paired samples t test Ordinal Nominal Wilcoxon Signed Ranks Chi Square Repeated measures ANOVA Friedman Chi square