Monitoring versus Improvement in CPV Jamey Crichton Principal

Monitoring versus Improvement in CPV Jamey Crichton - Principal Statistician May 23, 2017 www. cookpharmica. com

OVERVIEW - THEMES • Statisticians’ roles in CPV (Continued Process Verification) • How Efficient and Effective are our CPV programs and where do statistics impact these? • Monitor versus Improvement 2

WALL BETWEEN CLINICAL AND NON-CLINICAL STATISTICAL EXPERTISE Statisticians, Biostatisticians, Statistical Programmers Non-clinical Statisticians 3

1. ROLES FOR STATISTICIANS IN CPV • Cannot look at CPV in isolation – Stage 3 of Process Validation and interacts with many other processes. • FDA Guidance for Industry on Process Validation (2011) Ø Found at least 30 statements related to statistical support “Quotes from the Guidance Document” • Detect the presence and degree of variation • Analyze process and product data to evaluate the state of control • Sampling plan must result in statistical confidence • Application of suitable statistical procedures • An integrated team approach – expertise in statistics • Design of Experiments (DOE) studies can help • Validation of analytical methods • The data should be statistically trended by trained personnel • Recommend same sampling levels as PPQ (Stage 2) until sufficient data are available to generate significant variability estimates. 4

ARE WE UP FOR THE CHALLENGES ? • Saw this same phenomena back in the 80 s with the Auto industry with requirements placed upon suppliers by the Big 3. • Similar opportunities with the Six Sigma movement • Be willing to benchmark other industries • Be willing to study and understand the Business Processes surrounding the use of statistical methods • How much of our time/applications are related to compliance versus improvement? • Focus on the customers’ needs, our clients Ø Listen to them; they often are not sure what they need. Ø Most don’t really care about statistics. Ø They want efficient and effective ways to make decisions or meet compliance. 5

2. EFFICIENCY AND EFFECTIVENESS OF CPV PROGRAMS • Efficiency – how fast and resource intensive to get things done • Effectiveness – is it compliant and bringing value? • Do statistics affect E&E in CPV programs? YES Ø Ø Ø Ø EVERY investigation costs time and money – who pays for these? How “state of control” is defined Investigating all statistical signals, instead of practical ones Not understanding business processes Guidance on which tools to use, sampling plans Tools we bring to the table in investigations and improvement opportunities Sort out measurement variability versus process variability Opportunity to optimize MVDA approaches 6

HOW CAN WE INFLUENCE E&E • • • Investigate only practical signals – help to limit false signals Respond to signals as close to real-time as possible Connect CPV to APR reports Focus control charts on critical few – let RA drive this Utilize the Pareto Principle Every moment spent on one activity is time spent away from maybe more important activities. 7

3. MONITOR VERSUS IMPROVEMENT • What is meant by the word “Monitor” ? Ø “Monitor” is being used throughout the industry to mean control charting. • Auditors : “What is being monitored; show me the control charts”. • CPV monitoring plan – Variables either IN or OUT. • “We are monitoring all CQAs and a few IPCs and CPPs. ” • Interpreted as Binary, pass or fail, in or out. Ø This is very poor usage and needs to change. Ø Hospital example 8

MONITOR – FDA GUIDANCE ON PV (PAGE 7) “The terms attribute and parameter…are not categorized with respect to criticality in this guidance. With a lifecycle approach to process validation that employs risk based decision making throughout that lifecycle, the perception of criticality as a continuum rather than a binary state is more useful. All attributes and parameters should be evaluated in terms of their roles in the process and impact on the product or inprocess material, and reevaluated as new information becomes available. The degree of control over those attributes or parameters should be commensurate with their risk to the process and process output. In other words, a higher degree of control is appropriate for attributes or parameters that pose a higher risk. ” 9

HOW CAN THIS BE ACCOMPLISHED ? • • Benchmark other industries – already happening 30+ years ago Basic elements are already in place A few critical pieces are missing, inactive, or just not working well Establish this continuum of monitoring or Levels of Intensity, which applies to the actual “control” method and frequency/amount of sampling. Level 0 1 2 3 4 5 6 7 Monitoring Intensity or Control Level not looking or recording data in batch records comparing data to specifications run charts pre-control limits control charts automated feedback controller automated feedforward control 10

HOW CAN THIS BE ACCOMPLISHED ? Knowledge from Product & Process Development Risk Assessment Feedback; update Is this working? Control Strategy Performance PPQ/CPV 11

RISK ASSESSMENT Failure Modes & Effects Analysis Process/Product: FMEA Team: Black Belt: FMEA Date: (original) (Revised) Page: Text Number Text Number Risk Priority Number Detection Current Controls Occurrence Actions Potential Cause(s) of Failure Severity Potential Effects of Failure Potential Failure Mode Item Process Steps Process Calculation 0 0 0 0 0 Total Risk Priority: Resulting Risk 0 Priority 12

Risk Priority Number CONTROL PLAN Company X Calculation 0 0 0 0 0 Control Plan Description: Team Leader/Phone: Orig. Date: Part Number: Team Members Revision Level: Process Descriptions Customer Eng Approval/Date: Customer QA Approval/Date: Supplier/Location Process Machine and Description Tool Number Step Current Rev. Date: Supplier approval/Date: Product or Process Characteristic Risk: Major Average Minor Customer Other Approval/Date: Methods Specification and tolerance Measurement Method Sample Size Sample Frequency Control Method Reaction Plan Monitoring Intensity 13

EXAMPLES OF FILL WEIGHT 14

HOW CAN THIS BE ACCOMPLISHED ? Knowledge from Product & Process Development Feedback Risk Assessment Control Strategy Performance PPQ/CPV 15

WHAT IS THE VALUE OF MONITORING WITH CONTROL CHARTS ? • Certainly compliance – minimize compliance related aspects and work on continuous improvement; seek out significant opportunities. • How do control charts help on the improvement side? (Business Process) Ø For adjustment purposes, it is only maintaining the status quo. Ø Each signal is an opportunity to learn. 16

RESPONDING TO SIGNALS • Do we pursue these 2 learning opportunities? • As statisticians, we generally recommend chasing signals. • We always want to learn right? (internet example) • Let’s look at this from a Business Process standpoint Ø Ø What is the context ? (i. e. specs) What is the investment - time and resources to investigate? (Efficiency) What are the chances of finding root causes? (Effectiveness) What will have been gained? 17

IMPACT OF SPECIAL CAUSES Ppk = 3. 46 Cpk = 3. 64 Ppk = 3. 91 Cpk = 4. 16 What have we gained? Reduced risk of OOS from 0. 0000 XXX to 0. 0000 YYY 18

RESPONDING TO SIGNALS • • Control charts are not too valuable in this example. Tends to create over-reaction, including by auditors. This should be seen in the risk assessment and control plan. Maybe practical alarm limits are more appropriate. • In other industries, control charts are reserved for variables with lower Ppk’s (< 1. 5); large values of Ppk are seen as over-designed. 19

BIOLOGICS – SPECS BASED UPON CAPABILITY (+/- 3 SD) 20

MONITOR VERSUS CHANGE • George Box was famous for describing Passive Observation of processes versus Active Experimentation. He developed EVOP in the late 50 s as a way to experiment in production. Instead of waiting for significant changes, go create them. • “Every production process should not only generate product, but information on how to make it better the next time. ” • FDA provides this opening with the design space concept, allowing companies to explore the design space for optimization and improvement. • MVDA is an approach to attempt to utilize historical data. 21

Process Steps Where are the big $$ Losses/Wastes? MBR Set points and ranges Current Info Material properties Particle size Hold time Monitoring/Control Set-Up Disposition Sampling Intensity • Purpose • RA/FMEA – control level • Past performance Knowledge Feedback loop Past History Use to set-up Develop cheat sheet Study SOV “ A production process should not only produce product, but also information on how to improve it in the future. ” George Box - EVOP

REFERENCES • George Box and Norman Draper, Evolutionary Operation – A Statistical Method for Process Improvement, 1969, John Wiley and Sons. • Victor Kane, Defect Prevention, 1989, Marcel Dekker. • FDA Guidance on Process Validation, 2011. • AIAG PPAP Manual – Automotive Industry 23

Thank You ! 24