Diploma in Statistics Design and Analysis of Experiments

Diploma in Statistics Design and Analysis of Experiments Lecturer: Dr. Michael Stuart, Department of Statistics Office: LB 101 email: Michael. Stuart@tcd. ie Too short; cover parts 1 -3 in first half; need more for the 2 nd half. Try boys shoes, comparing two t-tests Diploma in Statistics Design and Analysis of Experiments 1

Design and Analysis of Experiments Course Outline • Experimental and observational studies • Basic design principles for experiments – Randomisation – Blocking (pairing) – Factorial structure • Standard designs, illustrated Diploma in Statistics Design and Analysis of Experiments 2

Design and Analysis of Experiments Course Outline • Analysis of experimental data – Exploratory data analysis – Parameter estimation and significance testing – Analysis of variance – Model validation, diagnostics • Computer laboratories • Strategies for Experimenting Diploma in Statistics Design and Analysis of Experiments 3

Design and Analysis of Experiments References Mullins, E. , Statistics for the Quality Control Chemistry Laboratory, Royal Society of Chemistry, 2003, particularly Chapters 4 -5, 7 -8. Detailed coverage of much of the module, in a specific context. Montgomery, D. C. , Design and analysis of experiments, 6 th ed. , Wiley, 2005. A comprehensive text, covers much more than this module, including statistical theory. Not always authoritative. Diploma in Statistics Design and Analysis of Experiments 4

Design and Analysis of Experiments Further reading Box, G. E. P, Hunter, J. S. and Hunter, W. G. , Statistics for Experimenters, 2 nd. ed. , Wiley, 2005. Includes many gems of wisdom from these masters of the genre, though not a course text. Daniel, C. , Applications of Statistics to Industrial Experimentation, Wiley, 1976. Includes many gems of wisdom from this master of the genre, using methodology appropriate for an industrial setting. Altman, D. G. , Practical Statistics for Medical Research, Chapman & Hall / CRC, 1991. Does what it says on the tin! Diploma in Statistics Design and Analysis of Experiments 5

Lecture 1. 1 1. Introduction to Course 2. Case study on process improvement statistical assessment of a process change strategy for experimentation 3. Experimental vs observational study another illustration 4. Multifactor designs efficiency interaction Diploma in Statistics Design and Analysis of Experiments 6

2 Case study on process improvement • Comparison of standard (old) and new processes for manufacture of electronic components • Key issues – homogeneity for valid comparison – systematic allocation – random allocation Diploma in Statistics Design and Analysis of Experiments 7

Experimental design • 50 components sampled per day, • 6 days per week, • 8 weeks, • Systematic layout, as follows Diploma in Statistics Design and Analysis of Experiments 8

Results Numbers of defectives per daily sample of 50 for 48 days (8 weeks) Diploma in Statistics Design and Analysis of Experiments 9

Comparison of two processes over eight weeks: data for first four weeks Diploma in Statistics Design and Analysis of Experiments 10

Comparison of two processes over eight weeks: data for last four weeks, with eight week summary Diploma in Statistics Design and Analysis of Experiments 11

Differences in numbers defective, with control limits No statistical significance! Diploma in Statistics Design and Analysis of Experiments 12

Alternative design (proposed by engineers) Assume this design was used; check for no effect Diploma in Statistics Design and Analysis of Experiments 13

Defect rates, per cent, with differences, for the first and second four week periods Diploma in Statistics Design and Analysis of Experiments 14

Defect rates, per cent, with differences, for the first and second four week periods highly statistically significant! Diploma in Statistics Design and Analysis of Experiments 15

Exercise Assess the statistical significance of the difference in defect rates, %, between the first period and second period for the old process. Homework Assess the statistical significance of the difference in defect rates, %, between the first period and second period for the new process. Diploma in Statistics Design and Analysis of Experiments 16

How can this be? Numbers defective in time order Long term downward trend, systematic bias Diploma in Statistics Design and Analysis of Experiments 17

How to avoid systematic bias • Make comparisons under homogeneous experimental conditions • 1 Systematic arrangement, as implemented: avoids known biases • 2 Random allocation: within each day pair, allocate old and new processes at random avoids known and unknown biases Diploma in Statistics Design and Analysis of Experiments 18

Two design principles • Blocking – identify homogeneous blocks of experimental units – assess effects of experimental change within homogeneous blocks – average effects across blocks • Randomisation – allocate experimental conditions to units at random Diploma in Statistics Design and Analysis of Experiments 19

Strategy for Experimentation The SIPOC Process Model Suppliers Inputs S I Outputs P O Customers Process C 20

Strategy for Experimentation Statistical Thinking Process Supplier Input measures Supplier performance Customer Inputs Outputs Process measures Process changes Output measures Customer Feedback Process management and improvement 21

Strategy for Experimentation Shewhart's PDCA Cycle Diploma in Statistics Design and Analysis of Experiments 22

Strategy for Experimentation Shewhart's PDCA Cycle • Plan: Plan a change to the process, predict its effect, plan to measure the effect • Do: Implement the change as an experiment and measure the effect • Check: Analyse the results to learn what effect the change had, if any • Act: If successful, make the change permanent, proceed to plan the next improvement or if not, proceed to plan an alternative change Diploma in Statistics Design and Analysis of Experiments 23

Strategy for Experimentation: new vs old manufacturing process Plan: • Compare defect rates for old process and new (cheaper) process – predict reduction, or no increase, in number of defectives using new process • Sample output over an eight week period, six days per week – select 50 components at random per day • Count number of defectives per sample Diploma in Statistics Design and Analysis of Experiments 24

Assessing experimental process for manufacturing electronic components Do: • Implement plan • Record daily numbers of defectives Diploma in Statistics Design and Analysis of Experiments 25

Assessing experimental process for manufacturing electronic components Check: • Analyse data • test statistical significance of the change Diploma in Statistics Design and Analysis of Experiments 26

Assessing experimental process for manufacturing electronic components Act: • If no worse, make the change permanent, – proceed to plan the next improvement or • if not, proceed to plan an alternative change Diploma in Statistics Design and Analysis of Experiments 27

3 Observational vs Experimental study Alternative design: • sample 1200 components from old process inventory, • sample 1200 components from new process inventory, • compare Diploma in Statistics Design and Analysis of Experiments 28

Example 2: walking babies • How long does it take a baby to walk? • Can this be affected by special training programs? 4 "training" programs: 1. special exercises 2. normal daily exercise 3. weekly check 4. end of study check each of 24 babies allocated at random to groups of 6 in each program. Diploma in Statistics Design and Analysis of Experiments 29

Example 2: walking babies Diploma in Statistics Design and Analysis of Experiments 30

Example 2: walking babies Alternative design: each of 4 different consultants prescribes one of the four training programs, select a sample randomly from babies assigned to each program. Problems: assignment of babies to programs equivelent to assignment of mothers to consultants lurking variables! Diploma in Statistics Design and Analysis of Experiments 31

Walking babies vs Defective components Level of control: less control means more variation Diploma in Statistics Design and Analysis of Experiments 32

4 Multi-factor experiments • Traditional versus statistical design – efficiency – interaction • Several levels • Several factors Diploma in Statistics Design and Analysis of Experiments 33

Illustration of a traditional design, with 12 experimental runs High Pressure Low (best) Low Diploma in Statistics Design and Analysis of Experiments Temperature High 34

Illustration of a full factorial design, with 12 experimental runs High Pressure Low Diploma in Statistics Design and Analysis of Experiments Temperature High 35

Interaction between the factors 75 best 60 15 High Pressure 5 Low best 65 5 5 Low Diploma in Statistics Design and Analysis of Experiments Temperature High 70 best 36

Multilevel Interaction: Emotional Arousal 160 subjects, – 80 male (M), – 80 female (F) shown one of 4 pictures: – – nude female, nude male, infant, landscape. Response variable: – level of emotional arousal Diploma in Statistics Design and Analysis of Experiments 37

Interaction between Factors Case study: Emotional Arousal Diploma in Statistics Design and Analysis of Experiments 38

Non-linear response: Optimisation vs Improvement Diploma in Statistics Design and Analysis of Experiments 39

Optimising performance; hill climbing Diploma in Statistics Design and Analysis of Experiments 40

Optimising performance; hill climbing Diploma in Statistics Design and Analysis of Experiments 41

Optimising performance; hill climbing Diploma in Statistics Design and Analysis of Experiments 42

Optimising performance; hill climbing Diploma in Statistics Design and Analysis of Experiments 43

Optimising performance; hill climbing Diploma in Statistics Design and Analysis of Experiments 44

Several factors 2 - level factors: 22 = 4 runs 3 factors: 23 = 8 runs 4 factors: 24 = 16 runs 5 factors: 25 = 32 runs 6 factors: 26 = 64 runs 7 factors: 27 = 128 runs Multi-level: 2 × 3 × 4 × 5 = 120 runs Diploma in Statistics Design and Analysis of Experiments 45

Reading SA Sections 1. 9, 11. 4 - 11. 6 EM Sections 4. 3, 4. 5. 1, 5. 2 DCM Section 2. 5, 3. 1 - 3. 3 Diploma in Statistics Design and Analysis of Experiments 46