Post Hoc Tests What is a Post Hoc

Post Hoc Tests

What is a Post Hoc Test? l Review: – – l Adjusting Alpha Level Multiple A Priori Comparisons What makes a test Post Hoc? – Many tests could be Post Hoc… But, there are set Post Hoc tests

Studentized Range Statistic q

Studentized Range Statistic q Independent Groups Largest mean Smallest mean Example 8. 2 11. 8 Fail to Reject Note: arrange means in ascending order! = 3. 77 critical value

Studentized Range Statistic q l q’s can tell us where differences are (more specific than F) l Solving q’s is just like solving t’s l But we solve a lot of q’s… can we speed things up?

Solving for the Smallest Significant Difference Example 8. 2 11. 8 = 3. 77 critical value

Solving for the Smallest Significant Difference l Solving for the smallest significant difference will help us make quicker comparisons l But we still need a way to organize things nicely…

Newman-Kewls smallest difference required was 6. 61 smallest significant difference If 2 steps Example T 1 8. 2 11. 8 T 2 T 3 r 0 3. 6 3 6. 61 3. 6 2 5. 41

A Better Newman-Kewls Example 2 3 3 9 10

A Better Newman-Kewls Example T 1 T 2 T 4 T 5 T 1 * * 3. 79 T 2 * * 3 3. 44 T 3 * * 2 2. 86 T 4 T 3 T 4 T 5 r 1 1 7 8 5 4. 04 0 6 7 4 6 7 1 T 3 T 4 T 5 Read Right to Left 1. 2. 3. T 1 T 2 UNTIL The row is completed A nonsignificant difference is found Reaching a column which was nonsignificant on the previous row T 2 T 3

Newman-Kewls Summarized l Newman-Kewls tables help organize your q’s l When doing a set of post hoc comparisons it’s best to use a Newman-Kewls table

Unequal N’s Tukey-Kramer Replace with

Unequal N’s Behrens-Fisher * * Each particular pairing of means must be examined with a different critical and their own value Thus, the smallest significant difference will vary even for a given

A Problem with q So Far… l Why are we doing q’s anyway? Why not do t’s instead? l But is q really controlling our alpha level? NO! Using q’s will give use more Type I Errors

Trying to fix q Tukey's HSD Tukey's WSD N-K except If there are 4 means, all differences are treated as 4 steps. r = # of steps between the two means to be compared. What Happens to Alpha Level? Power?

Tukey’s HSD and WSD T 1 T 2 T 3 T 4 T 5 r 1 1 7 8 5 4. 04 0 6 7 4 3. 79 6 7 3 3. 44 1 2 2. 86 T 5 Use Tukey’s WSD, not normal method for q

Back to Post Hoc in General l What is a post hoc test again? – l What are the real issues with Post Hoc tests? Alpha and Power… q is just one type of post hoc (one way to balance alpha and power), what are others?

Dunnett’s Control vs. Treatment run standard and use or, solve for critical difference (CV) Example Go to Table for * Pros…? Cons…?

Sheffé’s Test Linear contrast F= MS(contrast) To evaluate 1) consult F table and find critical value F. 05 (k 1, dferror) (CV) 2) multiply CV by (k-1). (new CV) MS(error) It sets the family-wise Type-I Error rate ( in our case) for ALL possible linear contrasts, not merely the pair-wise comparisons. Don’t use when only doing pair-wise, because it will be overly conservative.

Post Hoc Summary l When to use what… – q in most situations… but use Tukey’s WSD for critical value l l Put things in a Newman-Kewls table when N’s are unequal, use Tukey’s correction – Dunnett’s when you have one control and multiple treatments – Sheffé’s ONLY when you are doing complex comparisons (i. e. , contrasts)

Post Hoc Summary l Be aware of the alpha level and power issues… – Why can’t we have a perfect test (i. e. , low alpha level and high power)? – How does Tukey’s WSD and HSD relate to this? How does Dunnett’s relate to this? How does Sheffé’s relate to this? – –