Statistical Issues With Linear Extrapolation of Stability Data

Statistical Issues With Linear Extrapolation of Stability Data Jason Marlin, MS/T Statistics, Eli Lilly & Co. 1

Define OOT & PTE OOT—Out of Trend: the FDA considers any atypical test result or slope to be an “out of trend. ” PTE—Project to Exceed (Lilly term): linear extrapolation of stability data that indicates a lot may exceed the regulatory limit prior to expiry 2

FDA Resource for OOT PTE From Q 1 E: The Evaluation of Stability Data (June 2004) 3 The FDA expects industry to monitor stability batches and to react to trends or changes. Where appropriate, determine if batches may not stay within the regulatory specifications during shelf-life. Inform agency in advance if product will fail Modeling an appropriate fit is clearly important to this work.

Additional Resources � Published reference papers: 2003, Identification of Out of Trend Stability Results, by Ph. RMA CMC Statistics Stability Expert Teams, published in Pharmaceutical Technology � April Identification of Out-of-Trend Stability Results, Part II, by Ph. RMA CMC Statistics Stability Expert Teams, � October 2005, published in Pharmaceutical Technology 4 These papers establish three types of OOT data: 1) Analytical—a single result out of trend but within specification 2) Process Control—a succession of data points with an atypical pattern 3) Compliance—a single result or succession of results indicates the likelihood for an Out Of Specification prior to Expiry

PTE Resource From April 2003, Identification of Out of Trend Stability Results 5 “The extrapolation of OOT should be limited and scientifically justified, just as the use of extrapolation of stability data is limited in regulatory guidance (ICH, FDA). “

WIIFM Every pharmaceutical company is required to evaluate stability data for trends and respond accordingly. Every product has different challenges—e. g. assay (RMSE of common slopes), process mean, markets, possibly different stability timepoints, possibly different sites of manufacture, etc. Intent: provide some measure of accounting for these differences in the process of evaluating stability data for projection to exceed. 6 What is not covered is a method for controlling the Type II error rate—namely, failing to signal a batch that will exceed regulatory prior to expiry. (A rough simulation has been performed but further refinement is needed).

WIIFM-continued PTE’s are often evaluated by a stability coordinator (often non-statistician) A false signal=wasted resources (deviation, investigation, re-assay, etc. ) Generic computer algorithms don’t account for necessary variables (how many batches are manufactured, assay, process mean, etc. ) 7 Thus, some “decision-science” is needed to allow the coordinator to determine how many stability timepoints are required to obtain a reliable projection

Example: Actual Example ( w/coded data) 7 stability batches, all with data through at least 50% of expiry 8

Example: Actual Product w/coded data continued Expiry A lot with 6 timepoints PTE. 6 th timepoint at 18 months 9

Sources of OOT Reasons for a false signal: Non-linear data Limited data Process mean deviation from label claim Assay variability simulation What data requirements are needed to limit the probability of a false signal? Problem: for products with no practical change on stability and relatively high assay variability, three or four points are not sufficient to prevent false signals 10

Simulation Properties with Single-sided and Dual-sided limits are evaluated. 11 For single-sided limits, the mean bias and the assay standard deviation are expressed as a portion of the regulatory limit For dual-sided limits, the mean bias and the regulatory limit are expressed in actual units as % of label claim but they can also be expressed as k-sigma units of the assay standard deviation For the sake of simplicity, both single-sided and dual-sided properties are expressed in k-sigma units as “distance to nearest specification. ” This eliminates the need for presenting two sets of results.

Simulation-cont’d Two-sided Limit One-sided Limit 12

Simulation—continued Population means and assay standard deviations (assumed known) simulated relative to target. For single-sided and dual-sided limits, 1500 simulations are performed for 40 different combinations of true mean, regulatory limits and assay standard deviation Assumptions: 1) No Change on Stability 2) 36 -month expiry 3) Typical Stability timepoints (0, 3, 6, 9, 12, 18, 24, 30) A reduction in alpha (α)—all else held constant—could affect beta (β). **The assumption that there is not change on stability MUST be based on the science of the molecule/packaging and supported by available stability data 13

Errors in PTE Type I—designating a batch as PTE when the true property is not outside regulatory limits at expiry. Type II—failing to detect a batch with a true property value outside regulatory limits at or before expiry. 14

Goal In general, identify the minimum n such that no PTE’s are issued >95% of the time for properties which are practically stable. Also evaluated n (minimum # of stability timepoints) for 90 and 99%. Allow manufacturing sites to select a minimum # of stability time-points to achieve the desired alpha. Allow manufacturing sites to see the impact of mean bias and assay standard deviation on the likelihood of falsely PTE 15 Target products are those with large assay error and with process means close to the Regulatory limits (small R values)

Brief 16 JMP Simulation Overview

R Question: Is an R=4 the same for different combinations? R 1=4=100 -90/2. 5 R 2=4=99 -95/1 Simulation shows the results are effectively the same for a given n. 17

Rn=4(0, 3, 6, 9, 12, 18, 24 months) Each point represents a different combination of assay, process mean and regulatory. Bivariate Fit of % Not PTE By R n=4 RMSE=2. 79% 18

Rn=5(0, 3, 6, 9, 12, 18, 24 months) Bivariate Fit of % Not PTE By R n=5 RMSE=2. 16% 19

Rn=6(0, 3, 6, 9, 12, 18, 24 months) Bivariate Fit of % Not PTE By R n=6 RMSE=0. 99% 20

Rn=7(0, 3, 6, 9, 12, 18, 24 months) Bivariate Fit of % Not PTE By R n=7 RMSE=0. 31% 21

R (4, 5, 6 & 7 stability timepoints) 22

R Summary Tables 0, 3, 6, 9, 12, 18, 24 months 90% Table R ≤ 2. 4 2. 5 -3. 2 3. 3 -7. 4 >7. 5 n 7 6 5 4 99% Table R ≤ 4. 8 4. 9 -7. 5 7. 6 -9. 5 23 90% Table R ≤ 1. 9 2. 0 -2. 4 2. 5 -3. 3 >3. 4 n 7 6 5 4 95% Table R ≤ 3. 2 3. 3 -4. 8 4. 9 -9. 4 >9. 5 0, 3, 6, 12, 18, 24, 30 months n 7 6 5 R ≤ 2. 4 2. 5 -3. 2 3. 30 -4. 8 >4. 8 n 7 6 5 4 99% Table R ≤ 3. 3 3. 4 -7. 4 7. 5 -9. 4 >9. 4 n 7 6 5 4

Summary • Simulation provides a method to protect against falsely PTE for products which are stable • Increasing the number of data-points required to perform a PTE evaluation can—for instances where actual product stability changes occur—result in an increased likelihood of failing to detect a PTE for a product which will have a true property value outside the regulatory specifications • PTE evaluations need to account for assay variability, mean bias and the scientific basis of product stability. 24

Summary • Next Steps—simulate with linear slope to determine at what expense the reduction in false PTE’s (by increasing n) is achieved ØDetermine how the PTE changes as a result of true change on stability ØInitial look (slopes of 0. 05%/month and 0. 10%/month ) for 36 -month dating and for a product with ± 10% regulatory limits, as long as the mean was within 2% of target, these slope changes would not result in a true property value outside the regulatory limits. 25

Talking Points • Distance of projection to expiry is very important • Alpha should be a ƒn of # of lots manufactured and historical data • In general, 4 stability timepoints is marginal for 95% probability of not falsely PTE • Beta dependent on assumption of stability 26

Questions 27

One-Sided Simulation For different true, but unknown property mean values and for different true but unknown assay variabilitys, simulate the probability of PTE given the seven assumed stability time-points for two different stability evaluations: 1500 simulations are performed for 20 different combinations of true mean and assay standard deviation 0, 3, 6, 9, 12, 18 and 24 months (standard validation timepoints) 0, 3, 6, 12, 18, 24 and 30 months (alternative timepoint approach) Assay stdev levels=10, 20, 30, 40 and 50 % of regulatory Mean Bias Levels=0, 5, 10, 20 and 25 % of regulatory Regulatory Level is not a factor for the one-sided simulation as both variables are expressed in terms of a given regulatory limit (this is verifed by simulation to work for different regulatory limits) Assumptions: 1) No Change on Stability 2) 36 -month expiry 3) Typical Stability timepoints 4) A reduction in alpha (α)—all else held constant—could affect beta (β). 28

Two-Sided Simulation For different true, but unknown property mean values (in % of label claim but also easily expressed in k-sigma units from target) and for different true but unknown assay variabilities, simulate the probability of PTE given the seven assumed stability time-points for two different stability evaluations: 1500 simulations are performed for 40 different combinations of true mean, regulatory limits and assay standard deviation 0, 3, 6, 9, 12, 18 and 24 months (standard validation timepoints) 0, 3, 6, 12, 18, 24 and 30 months (alternative timepoint approach) Assay stdev levels=0. 50, 0. 75, 1. 00, 1. 25 and 1. 50 % of target Mean Bias Levels=0. 0, 0. 5, 1. 0, and 1. 5 % from target Regulatory Levels=± 5% from target, ± 10% from target Assumptions: 1) No Change on Stability 2) 36 -month expiry 3) Typical Stability timepoints 4) A reduction in alpha (α)—all else held constant—could affect beta (β). 29

Summary Table for One-sided and Two-sided One-sided V 1 90% Table R n ≤ 1. 9 7 2. 0 -2. 4 6 2. 5 -3. 3 5 >3. 4 4 One-sided V 2 90% Table R n ≤ 2. 4 7 2. 5 -3. 2 6 3. 3 -7. 4 5 >7. 5 4 Two-sided V 1 90% Table R n ≤ 2. 3 6 2. 4 -3. 5 5 >6. 7 4 Two-sided V 2 90% Table R n ≤ 3. 2 6 3. 3 -4. 7 5 >4. 7 4 95% Table R n ≤ 2. 4 7 2. 5 -3. 2 6 3. 30 -4. 8 5 >4. 8 4 95% Table R n ≤ 3. 2 7 3. 3 -4. 8 6 4. 9 -9. 4 5 >9. 5 4 95% Table R n ≤ 2. 3 7 2. 4 -3. 2 6 3. 3 -4. 5 5 >4. 5 4 95% Table R n ≤ 2. 7 7 2. 8 -4. 5 6 4. 6 -5. 7 5 >5. 7 4 99% Table R n ≤ 3. 3 7 3. 4 -7. 4 6 7. 5 -9. 4 5 99% Table R n ≤ 4. 8 7 4. 9 -7. 5 6 7. 6 -9. 5 5 >9. 4 30 4 99% Table R ≤ 3. 3 3. 4 -4. 5 4. 6 -6. 7 n 7 6 5 99% Table R ≤ 4. 0 4. 1 -5. 3 5. 4 -8. 5 n 7 6 5 >6. 7 4 >8. 6 4

Summary graphs for R by simulation 0, 3, 6, 9, 12, 18, 24 mon 0, 3, 6, 12, 18, 24, 30 mo 31