Useful Block Designs for Gage RR Studies for

Useful Block Designs for Gage R&R Studies for Measuring Total Analytical Variability Jyh-Ming Shoung*, Areti Manola, Yan Shen, Stan Altan *jshoung@its. jnj. com Midwest Biopharmaceutical Statistics Workshop May 19, 2015

Outline • Measurement System Analysis – Gage R&R – Design and Outputs – ICH Q 2(R 1) • • Case Study I – Dissolution Case Study II – Content Uniformity Case Study III – In Vitro Release Assay Summary

Measurement System Analysis – Gage R&R • Gage R&R design is a well known statistical design embraced by Six Sigma practitioners – Describes quantitatively how a process is performing. – To achieve the designation of a Six Sigma process, it must not produce more than 3. 4 defects per million opportunities. • The typical Gage R&R (Repeatability and Reproducibility) study consists of a factorial type design with blocking used to measure the total variability in a measurement system and its important components. Other designs can be used also, examples shown in the case studies. • Gage R&R studies have been used for investigating chemical analytical methods for the active pharmaceutical ingredient, uniformity across dosage units and dissolution.

ICH Q 2(R 1) Repeatability: Repeatability expresses the precision under the same operating conditions over a short interval of time. Repeatability is also termed intra-assay precision Intermediate Precision (IP): Intermediate precision expresses within-laboratories variations: different days, different analysts, different equipment, etc. Reproducibility: Reproducibility expresses the precision between laboratories (collaborative studies, usually applied to standardization of methodology).

Measurement System Analysis – Gage R&R • Total Variability = Process Variability + Gage Variability (Measurement Error) • Two Components of Gage Variability: Ø Repeatability – Short Term Variability Inherent precision of the device itself Ø Reproducibility – Long Term Variability due to different laboratories, operators, time periods, environments or in general, different conditions v Intermediate Precision (IP): Reproducibility without laboratories

CASE STUDY I - DISSOLUTION

Do. E for Dissolution Test Design Parameters: 1. 3 Batches: A, B, C 2. 4 sites/labs: G, I, L, T 3. 2 apparatus and 2 analysts per site • Experiment to be run in 6 days • On each day, each analyst will carry out 1 dissolution run, total 12 dissolution run per lab/site • Samples are assigned to 2 HPLC runs according to the table, total 12 HPLC runs per lab/site Responses: Dissolution measurements at 5, 10, 15, 20, 30, 45, 60 and 90 minutes are collected with analysis focusing on Q time of 30 minutes Day Diss. Run Analyst App 8 2 1 10 1 2 1 1 2 9 2 1 1 3 2 2 6 2 2 4 1 1 2 1 1 5 2 2 7 2 1 11 1 2 3 4 5 6 V 1 V 2 V 3 V 4 V 5 V 6 HPLC Run B C A - - - 1 - - - B A C 2 A B C - - - 2 - - - A C B 1 A B C - - - 4 - - - A C B 3 C A B - - - 3 - - - C B A 4 B C A - - - 5 - - - B A C 6 C A B - - - 6 - - - C B A 5 B C A - - - 8 - - - B A C 7 A B C - - - 7 - - - A C B 8 B C A - - - 9 - - - B A C 10 C A B - - - 10 - - - C B A 9 A B C - - - 12 - - - A C B 11 C A B - - - 11 - - - C B A 12

Data Plot for 30 Minutes Dissolution

Mixed Effects Model for Dissolution at 30 min § Fixed effects: Batch, Site § Random effects : Day(Site), Dissolution. Run(Day), HPLCRun*Dissolution. Run(Site*Day) yn(ijklm) i j Dk(i) El(k) Hlm(ik) n(ijklm) = dissolution of the nth vessel for jth batch tested in ith Site for lth dissolution run at kth day with mth HPLC run, = ith Site effect, = jth batch effect, = random effect of kth day within ith Site: N(0, D 2), = random effect of lth dissolution run in kth day: N(0, Ei 2), = random effect of mth HPLC run and lth dissolution run within the kth day and ith Site N(0, H 2), = residual error (vessel-to-vessel): N(0, ei 2). IP = D 2 + E 2 + H 2 + e 2 and Repeatability = e 2 Uncertainty (se) term associated with one reportable value = mean of six vessels = SQRT( D 2 + E 2 + H 2 + e 2 /6)

Table 1: Variance Components Estimates Variance Components Estimate Lower 95% Upper 95% Day(Site) 0. 26 0. 04 5. 04 E+05 Diss. Run*HPLC (Site*Day) 0. 96 0. 41 4. 25 Site G 6. 35 2. 72 28. 06 Site I 0. 83 0. 20 80. 83 Site L 10. 30 4. 62 39. 63 Site T 0 - - Site G 5. 77 4. 05 8. 90 Site I 3. 08 2. 13 4. 87 Site L 8. 60 6. 12 12. 98 Site T 6. 72 4. 80 10. 10 Diss. Run(Day) Residual Site

Table 2: Repeatability and Intermediate Precision (IP) Site G I L T 2*se for a Source Estimate % of Total Lower 95% Upper 95% Reportable Value Repeatability 5. 77 43 4. 05 8. 90 5. 84 IP (Total) 13. 35 100 7. 22 5. 04 E+05 Repeatability 3. 08 60 2. 13 4. 87 IP (Total) 5. 14 100 2. 78 5. 04 E+05 Repeatability 8. 60 43 6. 12 12. 98 IP (Total) 20. 13 100 11. 19 5. 04 E+05 Repeatability 6. 72 85 4. 80 10. 10 IP (Total) 7. 95 100 5. 25 5. 04 E+05 3. 21 7. 20 3. 06

Site as random effect and homogeneous across sites Table 3: Variance Components Estimates Variance Components Site Day(Site) Diss. Run*HPLC (Site*Day) Diss. Run(Day) Residual Estimate 8. 73 1. 23 Lower 95% 2. 70 0. 44 Upper 95% 147. 71 9. 89 2. 65 1. 54 5. 60 1. 57 6. 21 0. 61 5. 13 9. 56 7. 67 Table 4: Repeatability and Reproducibility Source Repeatability Reproducibility (Total) Estimate % of Total Lower 95% Upper 95% 6. 21 30 5. 30 7. 67 20. 39 100 10. 42 180. 44 2*se for a Reportable Value 7. 80 Uncertainty (se) term associated with one reportable value = mean of six vessels = SQRT[ (Site+Day+Disso_run+HPLC run) + Repeatability/6 ]

CASE STUDY II – PARTICLE SIZE

Design for Laser Diffraction Test Method for Particle Size • Four laboratories, 4 analysts and 7 instruments Site/Lab Analyst Instrument ID A JL MS-2 B DH MM 2 C KC EOP 049 EOP 224 PSE 002 D LD 005 LL M 300 Batch Info L 1 Target batch L 2 Larger particle size L 3 Smaller particle size L 4 Registration batch at target L 5 100 L Target batch L 6 100 L Smaller particle size • Six Batches • Design of Experiments performed by each combination of analyst and instrument within each lab Batch Day 1 Day 2 Day 3 L 1 Syringe 1 -2 -3 -4 Syringe 5 -6 / L 2 Syringe 1 -2 / Syringe 3 -4 -5 -6 L 3 / Syringe 1 -2 -3 -4 Syringe 5 -6 L 4 Syringe 1 -2 -3 -4 Syringe 5 -6 / L 5 Syringe 1 -2 / Syringe 3 -4 -5 -6 L 6 / Syringe 1 -2 -3 -4 Syringe 5 -6 Response: particle size d 50 (50% of particle size distribution, m) 14

Data Plot for d 50

Mixed Effects Model for Particle Size d 50 Site_App - combination of Site-Analyst and Instrument (7 combinations), Fixed effects : Site_App and Batch Random effects : Day(Site_App), Day*Batch(Site_App) and residuals. yl(ijk) = d 50 of the lth syringe of jth batch tested by ith Site_App at kth day i = ith Site_App effect j = jth batch effect Dk(i) = random effect of kth day within ith Site_App: N(0, D 2) Ekj(i) = random interaction effect of jth batch by kth day within ith Site_App: N(0, E 2) l(ijk) = residual errors: N(0, e 2). IP = D 2 + E 2 + e 2 and Repeatability = e 2 Uncertainty (se) term associated with one reportable value = SQRT( D 2 + E 2 + e 2 )

Table 5: Variance Components Estimates Variance Components Day(SITE_APP) BATCH*Day (SITE_APP) Residual Estimate Lower 95% Upper 95% 0 - - 0. 00047 0. 00006 3. 61 E+109 0. 06 0. 05 0. 08 Table 6: Repeatability and Intermediate Precision (IP) Source 2*se for a Estimate % of Total Lower 95% Upper 95% Reportable Value Repeatability 0. 0627 99 0. 05141 0. 07818 IP (Total) 0. 0632 100 0. 05147 3. 61 E+109 0. 5027

Site_App as random effect, Site_Apps homogeneous Table 7: Variance Components Estimates Variance Components SITE_APP Day(SITE_APP) BATCH*Day(SITE_APP) Residual Estimate 0. 00185 0 0. 00037 0. 06278 Lower 95% 0. 00045 9. 60 E-05 0. 05149 Upper 95% 0. 19354 8. 85 E+172 0. 07827 Table 8: Repeatability and Reproducibility Source Estimate Repeatability Reproducibility (Total) 0. 063 0. 065 2*se for a % of Total Lower 95% Upper 95% Reportable Value 97 0. 05 0. 08 0. 51 100 0. 05 8. 85 E+172 Uncertainty (se) term associated with one reportable value = SQRT((Reproducibility)

CASE STUDY III - BIOASSAY

Do. E for Bioassay Design Parameters: 1. 2 Analyst: 1, 2 2. 5 assay/run per Analyst 3. 2 Plates within each assay 4. 6 concentration levels (%) Ø 25, 50, 67, 100, 150 and 200 Ø 3 different concentrations within each plate Analyst Run 1 2 1 3 4 5 Response: Potency 1 2 2 3 4 5 Plate 1 2 3 4 5 6 7 8 9 10 25 X X X X X X 50 X X X Concentration (%) 67 100 X X X X X X X 150 X X X X X X 200 X X X X X

Data Plot for Bioassay

Mixed Effects Model for Potency • Fixed effects : Analyst, Concentration • Random effects : Run(Analyst), Plate(Run*Analyst) and residual (within-plate) ym (ijkl) = log of mth potency for jth concentration tested by ith Analyst in lth Plate at kth Run i = ith Analyst effect j = jth Concentration effect Rk(i) = random effect of kth Run by ith Analyst: N(0, R 2) Pl(ik) = random effect of lth plate within kth Run by ith Analyst: N(0, P 2) m(ijkl) = residual errors: N(0, e 2). IP = R 2 + P 2 + e 2 and Repeatability = e 2 Reportable Value defined as mean of 3 runs, 2 plates/run /analyst Uncertainty = SQRT( R 2/3 + P 2/6 + e 2 /6) (Format IP)

Variance Components, Repeatability, Intermediate Precision Table 9: Variance Components Estimates (in log scale) Variance Components Estimate Lower 95% Upper 95% Run(Analyst) 0. 0015 0. 0003 1. 4679 Plate(Analyst*Run) 0. 0039 0. 0017 0. 0159 Residual 0. 0019 0. 0012 0. 0032 Table 10: Repeatability and Intermediate Precision (in log scale) Source Estimate % of Total Lower 95% Upper 95% Repeatability 0. 0019 26 0. 0012 0. 0032 IP (Total) 0. 0073 100 0. 0032 1. 4871

Variance Components, Repeatability, Intermediate Precision Table 11: Repeatability and Intermediate Precision (in linear scale) Source % 2*%GCV Geometric % of Total Lower 95% Upper 95% Reportable Variance Value Repeatability 19. 9 26 13. 0 34. 0 IP (Total) 76. 4 100 34. 0 15627. 6 7. 8

Remarks 1. Application of blocking and Statistical Do. E is an efficient way to study analytical method performance for the purpose of assessing reproducibility, intermediate precision and repeatability of chemical analytical methods. 2. Do. Es permit a logical decomposition of the total variability into meaningful component parts. 3. Results are straight forward to interpret and have scientific meaning. 4. Inherent precision of the device itself cannot always be estimated because of the destructive nature of the test – confounded with dosage unit variation. 5. Some sources of variability may be estimated with insufficient precision. – Not possible at the development stage to estimate process variability due to small number of lots (make it a fixed effect). 6. Various statistical designs can form the basis for a Gage R&R study to gather information about additional factors. 7. Improves understanding and ability to support product specifications. 25