Effect Sizes Power Analyses for group Designs Effect

  • Slides: 15
Download presentation
Effect Sizes & Power Analyses for group Designs • • Effect Size Estimates for

Effect Sizes & Power Analyses for group Designs • • Effect Size Estimates for k-group ANOVA designs Power Analysis for k-group ANOVA designs Effect Size Estimates for k-group X 2 designs Power Analysis for k-group X 2 designs k-

k-BG Effect Sizes When you have more than 2 groups, it is possible to

k-BG Effect Sizes When you have more than 2 groups, it is possible to compute the effect size for “the whole study”. Include the F-value, the df (both for the effect and the error), and click the button for the type of design you have (BG or WG) However, this type of effect size is not very helpful, because: -- you don’t know which pairwise comparison(s) make up the r -- it can only be compared to other designs with exactly the same combination of conditions

k-BG Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at

k-BG Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at a time (pairwise comparisons)… The most useful effect sizes for k-group designs are computed as the effect size for 2 groups (effect sizes for pairwise comparisons) Since you won’t have F-values for the pairwise comparisons, you will use Computator to complete a 2 -step computation Using info from the SPSPS output d = (M 1 - M 2 ) / MSerror r = d² -----d² + 4

Pairwise effect sizes computation for k-BG designs For no therapy vs. weekly therapy …

Pairwise effect sizes computation for k-BG designs For no therapy vs. weekly therapy … Descriptives outcome variable -- larger scores are better N no therapy weekly therapy daily therapy Total 25 24 30 79 Mean 42. 1400 43. 9500 50. 9600 46. 2607 Std. Deviation 12. 06261 9. 39588 14. 59920 12. 80355 For a BG design be sure to press

k-WG Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at

k-WG Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at a time (pairwise comparisons)… Effect sizes for k-group designs are computed as the effect size for 2 groups (effect sizes for pairwise comparisons) Since you won’t have F-values for the pairwise comparisons, you will use Computator to complete a 3 -step computation Using info from the SPSPS output d = (M 1 - M 2 ) / (MSerror * 2) dw = d * 2 dw² r = ----- dw² + 4

Pairwise effect sizes computation for k-WG designs For no intake vs. mid … Descriptive

Pairwise effect sizes computation for k-WG designs For no intake vs. mid … Descriptive Statistics INTAKE MID FINAL Mean 26. 6200 33. 8900 30. 6200 Std. Deviation 8. 99399 9. 02793 8. 45050 N 48 48 48 For a WG design be sure to press

Determining the power you need. . For a 2 -condition design. . . •

Determining the power you need. . For a 2 -condition design. . . • the omnibus-F is sufficient -- retain or reject, you’re done ! • you can easily determine the sample size needed to test any expected effect size with a given amount of power For a k-condition design … • the power of the omnibus-F - isn’t what matters ! • a significant omnibus-F only tells you that the “two most different” means are significantly different • follow-up (pairwise) analyses will be needed to test if the pattern of the mean differences matches the RH: • you don’t want to have a “pattern of results” that is really just a “pattern of differential statistical power” • you need to assure that you have sufficient power for the smallest pairwise effect needed to test your specific RH:

k-group Power Analyses As before, there are two kids of power analyses; ; ;

k-group Power Analyses As before, there are two kids of power analyses; ; ; A priori power analyses • conducted before the study is begun • start with r & desired power to determine the needed N Post hoc power analysis • conducted after retaining H 0: • start with r & N and determine power & Type II probability

Power Analyses for k-BG designs Important Symbols S is the total # of participants

Power Analyses for k-BG designs Important Symbols S is the total # of participants in that pairwise comp n = S / 2 is the # of participants in each condition of that pairwise comparison N = n * k is the total number or participants in the study Example • the smallest pairwise effect size for a 3 -BG study was. 25 • with r =. 25 and 80% power S = 120 • for each of the 2 conditions n = S / 2 = 120 / 2 = 60 • for the whole study N = n * k = 60 * 3 = 180

Power Analyses for k-WG designs Important Symbols S is the total # of participants

Power Analyses for k-WG designs Important Symbols S is the total # of participants in that pairwise comp For WG designs, every participant is in every condition, so… S is also the number of participants in each condition Example • the smallest pairwise effect size for a 3 -WG study was. 20 • with r =. 20 and 80% power S = 191 • for each condition of a WG design n = S = 191 • for the whole study N = S = 191

Combing LSD & r … Cx mean M dif r Tx 1 M dif

Combing LSD & r … Cx mean M dif r Tx 1 M dif r Cx 20. 3 Tx 1 24. 6 4. 3 . 32 Tx 2 32. 1 11. 8* . 54 7. 5* . 41 * indicates mean difference is significant based on LSD criterion (min dif = 6. 1) Something to notice … • The effect size of Cx vs. Tx 1 is substantial (Cohen calls. 30 “medium effect”), but is not significant, suggesting we should check the power of the study for testing an effect of this size.

k-group Effect Sizes When you have more than 2 groups, it is possible to

k-group Effect Sizes When you have more than 2 groups, it is possible to compute the effect size for “the whole study”. Include the X², the total N and click the button for df > 1 However, this type of effect size is not very helpful, because: -- you don’t know which pairwise comparison(s) make up the r -- it can only be compared to other designs with exactly the same combination of conditions

Pairwise Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at

Pairwise Effect Sizes Just as RH: for k-group designs involve comparing 2 groups at a time (pairwise comparisons)… The most useful effect sizes for k-group designs are computed as the effect size for 2 groups (effect sizes for pairwise comparisons) The effect size computator calculates the effect size for each pairwise X² it computes

k-group Power Analyses As before, there are two kinds of power analyses; ; ;

k-group Power Analyses As before, there are two kinds of power analyses; ; ; A priori power analyses • conducted before the study is begun • start with r & desired power to determine the needed N Post hoc power analysis • conducted after retaining H 0: • start with r & N and determine power & Type II probability

Power Analyses for k-group designs Important Symbols S is the total # of participants

Power Analyses for k-group designs Important Symbols S is the total # of participants in that pairwise comp n = S / 2 is the # of participants in each condition of that pairwise comparison N = n * k is the total number or participants in the study Example • the smallest pairwise X² effect size for a 3 -BG study was. 25 • with r =. 25 and 80% power S = 120 • for each of the 2 conditions n = S / 2 = 120 / 2 = 60 • for the whole study N = n * k = 60 *3 = 180