Quality Systems Managing Human Error and CAPA Effectiveness

Quality Systems: Managing Human Error and CAPA Effectiveness John Shaeffer May 2008

Objectives n n n Why is "Not following SOPs" a top issue for the pharmaceutical industry How to know if your CAPA will be effective How to manage human error within systems

Top 10 Drug Observations (FDA) 2008 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Total # of Citations Citation Text 734 – Responsibilities and procedures in QC not in writing/ followed 629 – Written production & process control procedures not followed 532 – Control procedures not established which monitor/validate manufacturing 457 – Lab controls do not include appropriate test procedures 451 – No written procedures for production and process controls 423 – Testing and release do not include appropriate lab results 422 – Failure to fully review discrepancies to determine whether or not the batch has been thoroughly distributed 401 – Employees are not given training in operations/procedures being conducted 400 – Batch production and records do not include complete information 344 – Written procedures not established/followed for the cleaning and maintenance of equipment (including utensils) As of 02/11/2008

Revision Process: Routine Violations Undefined procedure structure/usage rules + No “on the spot correction” process + Slow normal revision cycle times + High frequency of internal revisions = ROUTINE VIOLATIONS

Execution Excellence Goals n Prevent EVENTS 4 Increase SOP adherence 4 Increase CAPA effectiveness n Increase efficiencies 4 Improve product cycle times 4 Decrease inadvertent errors

Addressing Events - CAPAs Typical Organizational Progression n Blind Eye Awareness Accountability Shared Responsibility

Consequences! Events vs. Errors

Error Types Latent vs. Active

Typical CAPAs for Personnel Events n Retrain n Counsel Isolated incident - no action required n n Increased supervision n Disciplinary action up to dismissal

Typical CAPAs for Procedural Events n Fix the procedure 4 Add a note or caution 4 Add another step to specifically address the event n Fix the person 4 Counsel 4 Retrain 4 No action 4 Increased supervision 4 Disciplinary action

Underlying Principles People: n n n Are FALLIBLE Achieve high levels of performance based largely on the ENCOURAGEMENT and REINFORCEMENT received from leaders, peers and subordinates Will exhibit behaviors that are influenced by ORGANIZATION PROCESSES and VALUES Error likely situations are: n Predictable n Manageable n Preventable Events can be prevented: n Understanding the reasons mistakes occur n Applying the lessons from past events

Human Performance = Results + Behaviors Flawed Defenses Organizational Issues Initiating Action Automobile Manufacturer Error Precursors James Reason Ph. D. , 1990

ORGANIZATIONAL CULTURE n n RESPONSE to inefficiency and error is related to organizational culture. An organization's culture is reflected by what it does: 4 Practices 4 Procedures 4 Processes n An organization's culture is NOT what it claims to espouse or believe in.

ORGANIZATIONAL CULTURE 3 Types: 1. PATHOLOGIC; the organization says Ø “We don't make errors, and we don't tolerate people who do. ” Ø likely to shoot the messenger 2. BUREAUCRATIC: write a new rule 3. LEARNING ORGANIZATION seeks to understand the broader implications of error Ron Westrum Ph. D. , 1984

Typical Procedure Progression - Bureaucratic Time Compliant Operations Actions allowed by plant procedures Events Actions required to perform the job Non Compliance

LEARNING ORGANIZATION n Actively seek flaws in systems n Messengers are trained and rewarded n Responsibility is shared n Failures lead to far-reaching reforms n New ideas are welcomed

Errors & Organizational Culture Creating an organizational culture that supports OPEN DISCUSSION of errors & near misses is perhaps the SINGLE MOST EFFECTIVE INTERVENTION. As such a culture is created, THE ERROR RATE WILL INCREASE, not because more are made, but because MORE ARE REPORTED.

Human Performance = Results + Behaviors Flawed Defenses Organizational Issues Initiating Action Automobile Manufacturer Error Precursors James Reason Ph. D. , 1990

Defenses n Physical barriers to control the process 4 Examples: - Policies, Procedures and Job Aids - Alarms, Warning signs, Labels and Floor Markings n Flaws in defenses 4 Promotes errors 4 Creates error likely situations

Procedure Reading Test

PARIS IN THE SPRING ONCE IN A A LIFETIME BIRD IN THE HAND

Procedure Reading Test What did you see?

PARIS IN THE SPRING ONCE IN A A LIFETIME BIRD IN THE HAND

Humans will …. not see what is there and see what is not there

1 Can you raed tihs? i cdnuolt blveiee taht I cluod aulaclty uesdnatnrd waht I was rdanieg. The phaonmneal pweor of the hmuan mnid; aoccdrnig to a rscheearch at camabrigde Uinervtisy, it dseno't mtaetr in waht oerdr the ltteres in a wrod are, the olny iproamtnt tihng is taht the frsit and lsat ltteer be in the rghit pclae. The rset can be a taotl mses and you can sitll raed it whotuit a pboerlm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. Azanmig huh? yaeh and I awlyas tghuhot slpeling was ipmorantt! 1 Cambridge University

Human Performance = Results + Behaviors Flawed Defenses Organizational Issues Initiating Action Automobile Manufacturer Error Precursors James Reason Ph. D. , 1990

Error Precursors TWIN Analysis Task Work Individual Nature Demands Environment Capabilities (human nature) 4 complicated vs. 4 lighting simple 4 noise 4 time 4 clothing constraints requirements 4 multiple steps at the same time 4 space 4 first time evolution 4 shortcuts 4 training 4 culture 4 how long since last performed 4 time into shift 4 personal affects 4 egos 4 perceived pressures

Human Nature Effects Add these numbers (without a calculator!) Take 1000 and Add 40 to it. Now add another 1000. Now add 30. Another 1000. Now add 20. Now add another 1000. Now add 10. What is the total?

Answer n Did you get 5000? 4 Great job! You probably followed the procedure usage rules discussed earlier. n n n You are not alone! Did you get 4010 or 410? If you said 4100 great job! 4 However, chances are you violated the procedure usage rules discussed earlier. n The decimal sequence confuses our brain, which always jumps to the highest decimals (100 s instead of 10 s).

Human Nature Effects Take 1000 and Add 40 to it. Now add another 1000. Now add 30. Another 1000. Now add 20. Now add another 1000. Now add 10. What is the total?

Human Performance = Results + Behaviors Flawed Defenses Organizational Issues Initiating Action Automobile Manufacturer Error Precursors James Reason Ph. D. , 1990

Operational Modes (Initiating Action) Knowledge Based High ER = 1 in 2 Rule Based Task Demands Low ER = 1 in 1, 000 NOTE: Error Rate (ER) Skill Based ER = 1 in 10, 000 Low Attention High

Importance of Operation Modes n Determines corrective action n Dictates procedure usage rules n Identifies error likely situations n Determines efficiency of execution

Operational Modes (Initiating Action) Knowledge Based High ER = 1 in 2 Rule Based Task Demands Low ER = 1 in 1, 000 NOTE: Error Rate (ER) Skill Based ER = 1 in 10, 000 Low Attention High

Human Performance = Results + Behaviors Flawed Defenses Organizational Issues Initiating Action Automobile Manufacturer Error Precursors James Reason Ph. D. , 1990

Performance Problem CAUSES 26 Leading Organizations Surveyed n Airlines 4 Delta Airlines 4 Jet Blue n Automotive n 4 BBC n Financial Services & Banking 4 American Express 4 Barclays Bank n n Results: 24. 3% Individual Industrial 4 Agilent Manufacturing 4 Caterpillar 4 Steelcase 4 Molex 4 Johnson Controls Pharmaceutical and Healthcare 4 Becton Dickson 4 Eli Lilly 4 Humana 4 Novartis Government 4 Navy 4 New York Fire Department 4 Social Security Administration Retail 4 Coffee Ben & Tea Leaf 4 Godiva 4 SAB Miller 4 Lexus n Media n Telecom and Technology 4 Nextell 4 IBM 4 SAS 4 SAP Tony O’Driscol Ph. D. , 2006 75. 7% Environmental

Reliance on Employee vs. CA Effectiveness HIGH Desired area for CAPA Effectiveness Low LOW Reliance on Employee High

CAPA Effectiveness: Targeting Human Performance Problems 100 Total Events 75% of events are outside of individuals control 100 events Assume corrective actions are 20% effective Number of events with effective corrective actions 25 events

Procedure Checklist: Procedure GMP-10052, Documentation Of GMP Practices Step Performed By: Checked By: 1 One action per step JS AP 2 Every step to successfully complete the task should be in the procedure JS AP 3 Procedure steps should be in the order that they are performed JS AP 4 Each procedure step should be 2 lines or less (exceptions allowed but rarely) JS 5 Only information critical for step implementation should be included within the step JS AP 6 Use note for location if a particular piece of equipment is hard to find AP 7 JS AP 8 Use cautions to delineate steps that need to be done together in a rapid fashion or if time is critical Use bullets or numbering in accordance with site standards JS AP 9 Use specific nomenclature for identifying components to be manipulated when possible JS AP 10 Place visuals close to the relevant text 11 The range of acceptable values for an entry should be next to or below the entry JS AP 12 Critical steps should be identified JS AP 13 One series of signoffs per step (performed by, checked by, verified by) JS 14 Batch record signatures, checked by and verified by should be aligned in separate columns AP 15 JS AP 16 Any cautions or important notes should be in the procedure before the information is required to be used Write the WARNING: statement to the person using the SOP JS AP 17 Align the second line of the warning statement with the text and not the word JS AP Performed By: _____John Shaeffer_____________ Checked By: _____Amy Peterson______________ Date: ____09/24/06____ Date: ____ 09/24/06____

Procedure Checklist: Procedure GMP-10052, Documentation Of GMP Practices Step Performed By: Checked By: 1 One action per step JS AP 2 Every step to successfully complete the task should be in the procedure JS AP 3 Procedure steps should be in the order that they are performed JS AP 4 Each procedure step should be 2 lines or less (exceptions allowed but rarely) JS 5 Only information critical for step implementation should be included within the step JS AP 6 Use note for location if a particular piece of equipment is hard to find AP 7 Use cautions to delineate steps that need to be done together in a rapid fashion or if time is critical JS AP 8 Use bullets or numbering in accordance with site standards JS AP 9 Use specific nomenclature for identifying components to be manipulated when possible JS AP 10 Place visuals close to the relevant text 11 The range of acceptable values for an entry should be next to or below the entry JS AP 12 Critical steps should be identified JS AP 13 One series of signoffs per step (performed by, checked by, verified by) JS 14 Batch record signatures, checked by and verified by should be aligned in separate columns AP 15 Any cautions or important notes should be in the procedure before the information is required to be used JS AP 16 Write the WARNING: statement to the person using the SOP JS AP 17 Align the second line of the warning statement with the text and not the word JS AP Performed By: _____John Shaeffer_____________ Date: ____09/24/06____ Checked By: _____Amy Peterson______________ Date: ____ 09/24/06____

How many missing initials?

Procedure Checklist: Procedure GMP-10052, Documentation Of GMP Practices Step Performed By: Checked By: 1 One action per step JS AP 2 Every step to successfully complete the task should be in the procedure JS AP 3 Procedure steps should be in the order that they are performed JS AP 4 Each procedure step should be 2 lines or less (exceptions allowed but rarely) JS 5 Only information critical for step implementation should be included within the step JS AP 6 Use note for location if a particular piece of equipment is hard to find AP 7 Use cautions to delineate steps that need to be done together in a rapid fashion or if time is critical JS AP 8 Use bullets or numbering in accordance with site standards JS AP 9 Use specific nomenclature for identifying components to be manipulated when possible JS AP 10 Place visuals close to the relevant text 11 The range of acceptable values for an entry should be next to or below the entry JS AP 12 Critical steps should be identified JS AP 13 One series of signoffs per step (performed by, checked by, verified by) JS 14 Batch record signatures, checked by and verified by should be aligned in separate columns AP 15 Any cautions or important notes should be in the procedure before the information is required to be used JS AP 16 Write the WARNING: statement to the person using the SOP JS AP 17 Align the second line of the warning statement with the text and not the word JS AP Performed By: _____John Shaeffer_____________ Date: ____09/24/06____ Checked By: _____Amy Peterson______________ Date: ____ 09/24/06____

Ready to do it again?

Procedure Checklist: Procedure GMP-10052, Documentation Of GMP Practices One action per step JS AP Every step to successfully complete the task should be in the procedure JS AP Procedure steps should be in the order that they are performed JS AP Each procedure step should be 2 lines or less (exceptions allowed but rarely) JS ___ Only information critical for step implementation should be included within the step JS __ Use note for location if a particular piece of equipment is hard to find. ___ AP Use cautions to delineate steps that need to be done together in a rapid fashion, if time is critical, or an error likely situation JS AP Use bullets or numbering in accordance with site standards JS AP Use specific nomenclature for identifying components to be manipulated when possible JS AP Place visuals close to the relevant text The range of acceptable values for an entry should be next to or below the entry JS AP Critical steps should be identified JS AP One series of signoffs per step (performed by, checked by, verified by) JS __ Batch record signatures, checked by and verified by should be aligned in separate columns ___ AP Any cautions or important notes should be in the procedure before the information is required to be used JS __ Write the WARNING: statement to the person using the SOP JS AP Align the second line of the warning statement with the text and not the word JS AP Performed By: John Shaeffer Date: 09/24/06 Checked By: Amy Peterson Date: 09/24/06

How many missing initials?

Procedure Checklist: Procedure GMP-10052, Documentation Of GMP Practices One action per step JS AP Every step to successfully complete the task should be in the procedure JS AP Procedure steps should be in the order that they are performed JS AP Each procedure step should be 2 lines or less (exceptions allowed but rarely) JS ___ Only information critical for step implementation should be included within the step JS __ Use note for location if a particular piece of equipment is hard to find. ___ AP Use cautions to delineate steps that need to be done together in a rapid fashion, if time is critical, or an error likely situation JS AP Use bullets or numbering in accordance with site standards JS AP Use specific nomenclature for identifying components to be manipulated when possible JS AP Place visuals close to the relevant text The range of acceptable values for an entry should be next to or below the entry JS AP Critical steps should be identified JS AP One series of signoffs per step (performed by, checked by, verified by) JS __ Batch record signatures, checked by and verified by should be aligned in separate columns ___ AP Any cautions or important notes should be in the procedure before the information is required to be used JS __ Write the WARNING: statement to the person using the SOP JS AP Align the second line of the warning statement with the text and not the word JS AP Performed By: John Shaeffer Date: 09/24/06 Checked By: Amy Peterson Date: 09/24/06

Operational Modes (Initiating Action) Knowledge Based High ER = 1 in 2 Rule Based Task Demands Low ER = 1 in 1, 000 NOTE: Error Rate (ER) Skill Based ER = 1 in 10, 000 Low Attention High

Predicted Results n First document 4 Speed – 7 seconds average 4 Success rate – over 85% n Second document 4 Speed – doubled in time (15 seconds) 4 Success rate – reduced significantly (under 10%)

Performance Model Flawed Defenses 80% Organizational Procedure Job Aid Alarm Policy Label Sign CA 5% Initiating Action Process & values System alignment Communication Behavior (Culture) Error Precursors Task Demands Work Environment CA – Corrective Action Individual CA 95% Organizational Issues 20% Individual Capabilities Nature (human nature) James Reason Ph. D. , 1990, modified Operation Modes - Skill - Rule - Knowledge

“Defense in Depth Model” : Anatomy of Events LATENT ERRORS Organizational issues: - Poorly Written Procedures - Failed or Nonexistent Programmatic Barriers - Ineffective Management Initiating Action ACTIVE ERRORS - Weak Skills - Failed or Nonexistent Barriers Organizational Barriers Management Barriers Flawed Defenses Human Fallibility Modified from Managing the Risks of Organizational Accidents, James Reason, Pd. D. 1997. Event

Corrective Actions Effectiveness More n ELIMINATE: Organizational Issues n ELIMINATE: Flawed Defenses & Error Precursors n Less PROVIDE: the Preventative Actions for individual who initiates the action

Execution Excellence n Nine different components 1. 2. 3. 4. 5. 6. 7. 8. 9. HPM Basic Concepts HPM Based Investigations Human Factoring Records Human Factoring Processes Error Prevention Techniques Preventive Actions Behavior Modification Organizational Issues Procedure Usage

QUESTIONS?