Comparison of Definitive Screening Designs with Fractional Factorial

  • Slides: 1
Download presentation
Comparison of Definitive Screening Designs with Fractional Factorial or Plackett-Burman Designs Philip Mc. Goff

Comparison of Definitive Screening Designs with Fractional Factorial or Plackett-Burman Designs Philip Mc. Goff Statistical Sciences, Glaxo. Smith. Kline Background and Overview Objective for 8 Factor Designs Objective for 6 and 10 Factor Designs Definitive screening designs (DSD), introduced in 2011, claim to be a better alternative to fractional factorial (FF) designs for screening designs. Among the reasons given: • Resolution IV FF completely confound some two factor interactions • FF with center points can estimate curvature, but unable to determine which factor is causing the curvature • FF limited to design size of a power of 2: 8, 16, 32, etc. Compare the performance of definitive screening designs (DSD) with fractional-factorial designs (FFs) for a range of effect sizes. Designs to compare are: Compare the performance of definitive screening designs (DSD) with Plackett-Burman designs (PBs) for a range of effect sizes. • 6 Factor: Compare DSD with PB, each with 1 center point • 10 Factor: Compare DSD with PB, each with 1 center point Example Alias Structure for Illustration 6 and 10 factor alias structure are similar, so only 6 factor is shown. 6 factor DSD is similar to 8 factor DSD. Want to compare DSD to FF using simulation to show the relative performance of each design • For 8 factors, similarly sized designs can be fairly compared. An 8 factor DSD has 17 runs, as does a resolution IV 28 -4 FF with 1 center point • Assess benefit of 2 additional points: Either 2 additional center points for FF and DSD, or 2 augmentation points for DSD using dummy factor Commonly encountered experiments with 6 or 10 factors can be a good option for DSD, since similarly sized resolution IV FF are not available. • 6 and 10 factor DSD have 13 and 21 runs, respectively • 6 and 10 factor resolution IV FF with 1 center point have 17 and 33 runs, respectively Can compare DSD to similarly sized Plackett-Burman (PB) designs • 6 and 10 factor PB with 1 center point have 13 and 21 runs, respectively Investigate the Size of the Effects Use model: Y = 20 + 2 A + 2 B - 2 C - 2 D + 2 BC + 2 A 2 + Error • Reduce the size of each effect by 0. 2 each iteration • Actual coefficients in table below. 6 Factor PB: Main effects and interactions partially confounded; Quadratic fully confounded Example Alias Structure for Illustration [A] = A+0. 33(-BC-BD-BE+BF+CD-CE-CF+DE+DF-EF) [BC] = BC+0. 33(-A-D-E-F-AD+AE-AF-DE+DF+EF) [AA] = AA+BB+CC+DD+EE+FF 28 -4 Res IV FF: Main effects no confounding; Interactions fully confounded: Quadratic fully confounded [A] = A [BC] = AE+BC+DF+GH [AA] = AA+BB+CC+DD+EE+FF+GG+HH 6 Factor: Percent of Times Model Correctly Includes Term in Model by Effect Size 8 Factor DSD: Main effects no confounding; Interactions and Quadratic partially confounded [A] = A [BC] = BC+0. 167(-AB+AC-BD+BE-BF+BG+BH+CD+CE-CF+CG-CH) +0. 667(AE-AG+DF+DG+EH+FH)+0. 367(-AA-DD+EE+FF+GG-HH) [AA] = AA+0. 367(-BC-BD-BE-BF-BG-BH-CD-CE-CF-CG-CH-DE-DF-DG -DH-EF-EG-EH-FG-FH-GH)+0. 190(BB+CC+DD+EE+FF+GG-HH) 9 Factor DSD: Main effects partially confounded; Interactions and Quadratic partially confounded [A] = A+0. 125(B+C-D-E+F+G-H) [BC] and [AA] similar to 8 factor DSD Percent of Times Model Correctly Includes Term in Model by Effect Size 10 Factor: Percent of Times Model Correctly Includes Term in Model by Effect Size Note that a number of different models were compared in other simulations. This model is illustrative of what is consistently seen with the all of the models investigated. Simulations used to compare designs of similar size • 1000 data sets generated with random error ~ N(0, 1) • Forward selection used to find model fit using AICc • Compare percent of times model terms are correctly included in the best fit model, denoted in graphs as Pct Sig Acknowledgements • Rick Lewis and Pedro Torres: original work comparing FF with DSD. • Greg Stockdale and Sonya Godbert: in ad hoc group looking at design and analysis issues. • Danae Williams: help with some of the simulation work in SAS. Conclusions for 8 Factor Designs Augmentation and Extra Center Points • DSD: Adding runs does not appear to have much benefit. • FF: Adding center points improves estimates, especially curvature. Effect Size • Linear Effects: FF and DSDs perform about the same. • Interactions: FF has the best chance of finding significant interactions. Any interaction detected will be aliased with other interactions, though it is often relatively straightforward to determine which is the most likely interaction through expert knowledge. • Quadratic Effects: Adding center points to the FF shows the greatest improvement in detecting curvature, though the FF does not identify which factor is causing the curvature. Adding either center points or augmentation points does not improve DSDs. Overall Conclusions For 8 factor designs, a FF is preferred to a DSD. • FF: though interactions and quadratic effects are completely confounded, there is a higher probability of determining that a significant term is in the model. • DSD: complicated partial confounding of interactions and quadratic effects reduces the ability to determine that significant terms are in the model. For 6 or 10 factor designs, a DSD is preferred to a PB. • PB: the correlation between the main effects and interaction terms can greatly compromise the performance. • DSD: main effects are free of correlations with interactions which makes for better estimation of main effects when interactions are present. Created by: Digital Media Services