Human Factors Aspects in PostMaintenance Flight Tests Capt

Human Factors Aspects in Post-Maintenance Flight Tests Capt. Claudio D. Caceres MSc, MAIAA Academic Project for the award of the MSc in Air Safety Management, at the City University of London, UK. Rome, 26 March 2008 Continuous Security©

Agenda • Today’s programme • Brief description of my background • Outline - Introduction of the research’s objectives • Research methodology • Findings and Recommendations 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 2

Programme • Capt. Claudio D. Caceres presentation: 20 minutes; • Q&A; • Dr. Steve Bond, Ph. D/City University of London presentation: 20 minutes; • Q&A; 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 3

My background • • 1982 Started flying in Argentina 1995 DC-9/30 Alitalia pilot’s Training 1996 DC 9 -51 copilot in eurofly Sp. A, 1997 Co-pilot BAe 146/RJ/Then Captain Upgrade training, in AZZURRA Spa. 2000 A 320 Captain Volare Airlines Sp. A 2001 ISO 9001 -2000/Quality Auditor Training on behalf of Volare Airlines; 2001 MSc in Air Transport Management City Univ. Of London, UK; 2002 A 330 Captain Volare Airlines Sp. A; 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 4

My background • • • 2004 EASTI, Bruxelles, ECAC/ICAO Avsec Courses 2004 Qualified ECAC Avsec Certified Auditor 2004 Part Time ENAC advisor on Avsec matters • 2005 A 320 Captain, Air Malta Plc, UK Operations • 2006 B 737, Ryanair Captain • 2007 Got Married and started my own organisation CS • 2008 MSc in Air Safety Management, City Univ. of London 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 5

Outline • • Post Maintenance Flight Tests; Human Factors (HF) Aspects; Research methodology; Haz. Ident. & Risk management approach; Human Factors fundamentals Proposed Strategy to conduct safe FTs; Recommendations. 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 6

Post Maintenance Flight Tests • Necessary to confirm aircraft airworthiness performance data • Risky activity • No repetitive activity, no frequent data • Performed by highly qualified staff • Performed in busy skies • Improvisation increases the risk 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 7

Flight Tests - Objectives • Flight testing is conducted to ensure that the overall performance or handling qualities of an aircraft have not fallen below a defined standard. • After certain fault rectification or replacement of components when checks for properation cannot be carried out on ground. 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 8

Flight Tests - Objectives • To deliver safe Flight Tests operations and meeting the technical objectives. Risk Control Measure 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 9

Flight Tests – HF Objectives • To deliver safe Flight Tests operations and meeting the technical objectives. - CRM, RISK Control Measure 26/03/08 -Stress, Fear, Exultation. -Training, -Part A, 8. 7, - FTS, - Briefing. H. F. Aspects in post-Maintenance Flights. Continuous Security© 10

Safe Operations - Objectives • Controlling ”the rising risk” ATC Weather Pilots Part 145 CAMO 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 11

HF aspects • • Pilots are asked to deliver their best Non-cyclical-activity Excessive reliance on “experienced” staff Provision of clear policy, procedures and practices is essential • Planning is a key factor 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 12

Scope of the Test studyis a. . . Actually a Flight • . . . guess? 26/03/08 • or. . . systemic activity? CAMO PH COORDIN ATION PROCES S Part 145 CREW BRIEFI NG DATA GATHERI CAMO PH PH FO OUTPUT H. F. Aspects in post-Maintenance Flights. NG Continuous Security© 13

Research Methodology The following topics were reviewed Data gathering Fundamentals of HF Reviewing the Human Limitations to FT Risk Management Hazard Identification, Risk awareness and system protection Actual Legislation EASA and NAA Rules, Regulations and good practices Industry Approach assessment to FT How the industry is actually managing the issue Focus on system safety and on risk Identification and management. R. Sumwalt (2007), vice chairman of the NTSB, likened the hunt for precursors to "reading tea leaves" because it can require imagination to tie together incidents that don't seem hazardous at first blush. 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 14

Human Factors - Fundamentals Study • Peoples interacting with machines • In complex hazardous systems Safety & Efficiency • Human Performance & Reducing Human Error Optimisin g Principles & Methods • From behavioural & social sciences, physiology & Engineering Source: GAIN Book, Issue 2, 2001 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 16

Human Factors - Fundamentals Causes 100% ↑ 80% M es aus an C Machine C auses 1903 2008 → Source: FAA 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 17

HF - Systemic Vulnerability ORGANIZATIONAL INFLUENCES Resource Management Organizational Climate Organizational Process Source: FAA 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 18

HF – Unsafe Supervision UNSAFE SUPERVISION Inadequate Supervision Planned Inappropriate Operations Failed to Correct Problem Supervisory Violations Source: FAA 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 19

HF–Preconditions for Unsafe Acts Substandard Condition for Operators Adverse Mental States Adverse Physiological States Substandard Practices of Operators Physical/ Mental Limitations Crew Resource Management Personal Readiness Source: FAA 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 20

Human Factors – Shell Model The Shell Model Modified by Hawkins. H S S = Software (procedures, symbology, etc. ) H=Hardware (machine) L E=Environment L=Liveware (human) Source: ICAO Doc 9859, SMM. 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 21

HF– Basic Emotions Theorist Suggested Basic Emotion • Plutchik: • Arnold adds: • Panksepp adds: • Frijda adds: • aversion, courage, dejection, despair, hate • panic • happiness, interest, wonder, sorrow • Gray adds: • Rage, terror, anxiety • Izard adds: • contempt, distress, guilt, shame • Mc. Dougall adds: 26/03/08 • Acceptance, anger, anticipation, disgust, joy, fear, sadness, surprise, love • elation, subjection, tender-emotion H. F. Aspects in post-Maintenance Flights. Continuous Security© 22

HF–“Cockpit” Emotions (Source: Dickerson, Anheuser & Busch) Acceptance Admiration Elation Pride Relief Resolve Amazement Anticipation Interest Surprise 26/03/08 H. F. Aspects in post-Maintenance Flights. Panic Sadness Shame Sorrow Continuous Security© 23

Pilot Mental States (Dickerson) Nervous Confused Frustrated Rushed Task Saturated Intimidated Bored Disrespected or Insulted Self-Righteous Contemptuous Panic Impatience 26/03/08 H. F. Aspects in post-Maintenance Flights. Harassed Excited Distracted “Forgetting Something” Inadequate “Hidden Agenda” Threatened Muzzled Smug Defiance Continuous Security© 24

HF – FAA’s 5 Hazardous Attitudes Hazardous Attitude 26/03/08 Antidote Anti-authority: Don’t tell me what to do. Follow the rules. They are usually right. Impulsivity: Do something quickly. Not so fast. Think first. Invulnerability: It won’t happen to me. It could happen to me. Macho: I’m tough enough. I can do it! Taking chances is foolish. Resignation: What’s the use? I’m not helpless. I can make a difference. H. F. Aspects in post-Maintenance Flights. Continuous Security© 25

Pilots: The Flight Test Rules Guidance: Flight Test Schedule(FTS) SOPs 26/03/08 AFM H. F. Aspects in post-Maintenance Flights. Dedicated Manufacturer Chapter Continuous Security© 26

Flight Test Schedule - Risk Guidance: Flight Test Schedule(FTS) SOPs 26/03/08 AFM H. F. Aspects in post-Maintenance Flights. RISK Manufacturer Test Schedules Continuous Security© 27

Risk Assessment: Task – Rule boundary 26/03/08 AFM 4% of FT Dedicated Manufacture’s Schedule 1% of FT SOPs 95% of FT AFM 4% of FT H. F. Aspects in post-Maintenance Flights. Continuous Security© 28

Risk Assessment – Time factor Dedicated Manufacture’s Schedule 1% of FT AFM nt e 4% of FT em r i u q No t s : . 1 e R S FT Te 2: . o nt e m S FT ire u q Re N st e T SOPs 95% of FT 26/03/08 nt e m S st e T FT : . 3 o N ire u q e R H. F. Aspects in post-Maintenance Flights. AFM 4% of FT Continuous Security© 29

Risk Assessment – Severity Does the operator know: Dedicated If there is still enough Manufacture’s protection? Schedule Is the 1% risk still acceptable? of FT AFM nt e 4% of FT em r i u q No t s : . 1 e R S FT Te 2: . o nt e m S FT ire u q Re N st e T SOPs 95% of FT 26/03/08 nt e m S st e T FT : . 3 o N ire u q e R H. F. Aspects in post-Maintenance Flights. AFM 4% of FT Continuous Security© 30

Risk Assessment – Policy Does the operator know: Dedicated How to exit? Manufacture’s Schedule Who can say STOP! How to 1% recover? of FT AFM nt e 4% of FT em r i u q No t s : . 1 e R S FT Te 2: . o nt e m S FT ire u q Re N st e T SOPs 95% of FT 26/03/08 nt e m S st e T FT : . 3 o N ire u q e R H. F. Aspects in post-Maintenance Flights. AFM 4% of FT Continuous Security© 31

Manoeuvre & control measure Normal • SOPs – Flight Crew is Familiar Close to aircraft limitations • AFM – Flight Crew Self familiarization required • (NOTE: Often AFM is not OM Part B) Beyond aircraft limitations • Dedicated Manufacturer’s Chapter in AFM • Provision to Flight Crew of a dedicated training/SIM session 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 32

Survey Questionnaire sent to 57 AOC’s Italian Operators: • 10 responded • 1 declined to participate • 1 provided informal data • After a cleaning procedure the assessment is based on No. 8 Operators Question Number Yes No N/A No. 1 (Policy for flight tests) 6 (75%) 2 (25%) 0 No. 2 (If it includes Human Factors aspects) 3 (37, 5%) 5 (62, 5%) 0 No. 3 (Identification of Risks) 4 (50%) 0 No. 4 (Involvement in Safety occurrences) 4 (50%) 0 No. 5 (Implementation of corrective action) 4 (50%) 3 (37, 5%) 1 No. 6 (Monitoring of effectiveness of the corrective 5 (62, 5%) 3 37, 5%) 1 actions) 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 33

Findings from Docs assessment Confidentially the author reviewed No. 10 OM Part A/Ch. 8. 7 AOC’s Holders • 5 were Italian AOC • 5 were EU AOC 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 34

The organization must provide Policy Procedures Safety Culture Practices People Source: City University, Safety by Design Module Course notes. 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 35

Finding No. 1: Define the owner of the process PHFO is the owner of the process Responsibility Hazard Identification & Risk assessment (Process Sharing) Process Execution 26/03/08 Quality & Safety Management System FOPH Flight Crew Composition H. F. Aspects in post-Maintenance Flights. CAMO PH Engineer Members from the Authority Continuous Security© 38

Six varieties of rule-related performance Good Rules Correct Bad Rules No Rules Correct Performance compliance violation Improvisation Erroneous Misvention Mispliance Mistake Performance Source: J. Reason (1997). 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 39

Total time required for the process A generic example of Risk assessment in the Process of F. T. On-blocks 26/03/08 Hazards indentif. /Risk D Assessment Permit to Fly Off-blocks F. S. R. B Part 145/CAMO PH Flight Crew Composition FOPH Technical Briefing/FTS CAMO PH Flight Planning Dispatch/FOPH Weather Minima Operating CPT ATC Co-ordination and Call sign Dispatch/CPT Flight Crew Briefing Safety Accurate Pre-flight Inspection D CPT to Engineering+ NAA Member Flight Crew D Flight Crew Aircraft acceptance D D + Cabin safety Ramp Safety Flight Crew ATC Clearance Both Pilots headsets Execution of the flight Flight Crew Collection of data/FTS Flight Crew Landing + Ramp Safety Flight Crew De-briefing/Paper work review CPT to CAMO PH + NAA Feedback for improvement D D wearing D D Captain to FOPH H. F. Aspects in post-Maintenance Flights. Continuous Security© 40

Which are the main risks? • • HF of staff: exposure to emotional triggers The combination of an error + a violation Ramp activity ATC Working area Other FT crew or members Secondary failures 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 41

Actual Legislation ICAO Annex 8 Airworthiness JAR/OPS 1/3 (Reg. EU 8/08 EUOPS from 16/07/08) Sub-part P/ O. M. Contents 8. 7 Non-Revenue FLTs Procedures & Limitations b) Flight Test 26/03/08 EASA State of Register Reg. EU 2042/2003 Sub-part P/ O. M. Contents AFM: Chapter Regarding Flight Test MA. 706 Staff involved (List of Pilots) Part 21 Airworthiness Review TC/STC Manufacturer Maintenance Manual Flight Test Schedule H. F. Aspects in post-Maintenance Flights. Continuous Security© 42

Actual AOC Holder Documentation AOC Operator OPS OM Part A/ Chapter 8. 7 Part M OM Part B + AFM: Chapter Regarding Flight Test Maintenance Manual CAME List of Pilots Procedure for Flight Tests Dedicated Chapter Flight Tests Manufacturer Flight Test Schedule 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 43

Recommendation Finding Action Define clear policy on FT Part A, 8. 7 – Policy Define clear FT Procedures Part A, 8. 7 – Procedures Define a FT pilot qualification process Part D – Qualification & Training Define a list of pilots (EASA/M. A. 706) CAME – FT Chapter Define clear FT Practices Part A, 8. 7 – Mandatory briefing Consider HF aspects of FT Part A, 8, 7 – HF aspects during FT Review Instructions to FT crew CAME – Flight Test Schedule (FTS) Define possible FT fleet wise CAME – FT Requirement Review & up-to-date risk assessment F. S. R. B. : FOPH, CAMO PH, QM, SM Define recurrent training Part D – FT Recurrent training 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 44

Summary • Key points to deliver a safe post-M. F. T. • The requirement to establish documented procedures for F. T. • The H. F. aspects during post-M. F. T. • The hazard and risk approach • Safety aspects that can help the organisation in delivering safer post-M. F. T. 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 45

Supportive Documentation • Books: – – – – 26/03/08 ICAO Doc. 9859, SMM. UK CAA, Casey, S. (1993). Set Phasers on Stun and Other True Tales of Design, Technology, and Human Error. Santa Barbara, CA: Aegean Publishing Company. Dekker, S. (2002). The Field Guide to Human Error Investigations. Aldershot, Hampshire GU 11 3 HR: Ashgate Publishing Limited. Dörner, D. (1996). The Logic of Failure: Recognizing and Avoiding Error in Complex Situations. New York, N. Y. : Metropolitan Books. Gigerenzer, G. (2003). Reckoning with Risk: Learning to Live with Uncertainty. London, UK: Penguin Books Ltd. Hawkins, F. , Orlady, H. (2001). Human Factors in Flight. (Second ed. ). Aldershot, Hants GU 11 3 H: Ashgate Publishing Limited. Helmreich, R. L. , Merritt, A. C. (2001). Culture at Work in Aviation and Medicine: National, Organizational and Professional Influences. Aldershot, Hants GU 11 3 H: Ashgate Publishing Limited. Maurino, D. , Reason, J. , Johnston, N. & Lee, R. (1995). Beyond Aviation Human Factors. Aldershot, Hants GU 11 3 H: Ashgate Publishing Limited. Orasanu, J. , Fischer, U. & Davison, J. (2004). Risk Perception and Risk Management in Aviation. In Dietrich, R. and Jochum, K. (Eds. ), Teaming Up: Components of 69 Safety under High Risk (pp. 93 -116). Aldershot, Hants GU 11 3 H: Ashgate Publishing Limited. Patankar, M. , Taylor, J. (2004). Risk Management and Error Reduction in Aviation Maintenance. Aldershot, Hants GU 11 3 H: Ashgate Publishing Limited. Perrow, C. (1999). Normal Accidents: Living With High-Risk Technologies. Princeton, New Jersey: Princeton University Press. Reason, J. , Hobbs, A. D. (2003). Managing Maintenance Error; A Practical Guide. Aldershot, Hants GU 11 3 H: Ashgate Publishing Limited. Reason, J. (1997). Managing the Risks of Organizational Accidents. Aldershot, Hants GU 11 3 H: Ashgate Publishing Limited. Reason, J. (1999). Safety Paradoxes and Safety Culture. Injury Control and Safety Promotion 7(1), 3 -14. Strauch, B. (2004). Investigating Human Error: Incidents, Accidents, and Complex Systems. Aldershot, Hants GU 11 3 H: Ashgate Publishing Limited. H. F. Aspects in post-Maintenance Flights. Continuous Security© 46

Supportive websites • Web sites: – The CAA Flight Check Handbook: http: //www. caa. co. uk/default. aspx? catid=1455&pagetype=90&pageid=8913 – The CAA Flight Test Schedule: http: //www. caa. co. uk/default. aspx? catid=1455&pagetype=68 – The FAA HF http: //www. hfskyway. com/ - ENAC Permit to fly: - http: //www. enac-italia. it/documents/download/nor/NAV-32 B. pdf - ENAC Permit to fly for non-ENAC Resgisterd aircrafts: - http: //www. enac-italia. it/documents/download/nor/NAV_62 A. pdf 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 47

Thank you • Questions? 26/03/08 H. F. Aspects in post-Maintenance Flights. Continuous Security© 48

Contacts Contatti Dr. Steve Bond, Ph. D, Course Director MSc in Air Safety Management, City University of London. Com. te Claudio D. Caceres, www. city. ac. uk/sems/postgraduate/asm/ Italy Phone: +44 (0) 20 7040 8926 Phone/fax: +39. 035. 499. 71. 00 Fax: +44 (0) 20 7040 8566 Email: s. j. bond@city. ac. uk 26/03/08 MSc, Member of AIAA. Continuous Security® Via Tasso, 6 – 24040 Filago (BG) Mobile: +39. 348. 26. 89. 715 c_d_caceres@infinito. it www. mycs. it H. F. Aspects in post-Maintenance Flights. Continuous Security© 49