Commercial Aviation Safety Team CAST Overview In the

Commercial Aviation Safety Team (CAST) Overview

In the U. S. , our focus was set by the White House Commission on Aviation Safety, and The National Civil Aviation Review Commission (NCARC) 1. 1. . . Reduce Fatal Accident Rate. . . • . . . Strategic Plan to Improve Safety. . . • . . . Improve Safety Worldwide. . .

Vision - Mission - Goals Vision n Key aviation stakeholders acting cooperatively to lead the world-wide aviation community to the highest levels of global commercial aviation safety by focusing on the right things. Mission n Enable a continuous improvement framework built on monitoring the effectiveness of implemented actions and modifying actions to achieve the goal. Goal n n Reduce the US commercial aviation fatal accident rate 80% by 2007 and Maintain a continuous reduction in fatality risk in US and International commercial aviation beyond 2007.

CAST brings key stakeholders to cooperatively develop & implement a prioritized safety agenda Government Industry AIA Airbus ALPA ATA IFALPA NACA Boeing GE* RAA FSF Commercial Aviation Safety Team (CAST) IATA** AAPA** ATAC** APFA** ACI-NA** * Representing P&W and RR ** Observer DOD FAA • • • Aircraft Certification Flight Standards System Safety Air Traffic Operations Research NASA ICAO** EASA (ECAST) TCC NATCA** NTSB**

1. NTSB Accident Incident Reports Safety Analysis Process 2. 21. 3 Reports ASIAS data Airclaims data Industry Historical Data Pareto Plots JSAT 3. 4. Intervention Strategy Accident Threat Combined Threat Causal Analysis Cause 5. 6. JSIT Cause JSAT Implementation Strategy Cause Safer Skies Industry 7. Government Av. SP Coordinated Plan Measuring Progress to Goal 5. 3 -23

CAST Safety Strategy Data Analysis Implement Safety Enhancements - U. S. Set Safety Priorities Agree on problems and interventions Achieve consensus on priorities Influence Safety Enhancements Worldwide Integrate into existing work and distribute

Commercial Aviation Safety Team (CAST) CAST Joint Safety Analysis Teams (JSAT) • Data analyses Joint Safety Implementation Teams (JSIT) • Safety enhancement development Joint Implementation Measurement Data Analysis Team (JIMDAT) • Master safety plan • Enhancement effectiveness • Future areas of study

Joint Safety Analysis Team (JSAT) Process

Typical JSAT Membership • • • ALPA/APA FAA (AIR, AFS, ASA, AAI, ATO) Airbus EASA ATA Transport Canada • • NASA Engine companies – (PW, GE, RR-Allison) Boeing RAA NACA AIA NATCA

JSAT Process Charter Development Evaluate Problem Importance Global Review of Characteristics/ Indicators Assign Standard Problem Statements Identify Intervention Strategies Establish Team Identify Problems (what/why) Evaluate Intervention Effectiveness Select Data Set Develop Event Sequence Prioritize Interventions Review Data Record Characteristics/ Indicators Technical Review & Report Results 5. 5 -24

Developed Event Sequence • Facts and data • Pilot - controller voice events • Missed calls • Events that occurred or should have • Time coded each event

Develop Problem Statements • Problem statements – – What went wrong Deficiency definition Potential reason Something which happened or didn’t happen

Sample Standard Problem Statements • 10 FLIGHTCREW – Failure of flight crew to follow established procedures (SOP) • 39 AIRCRAFT EQUIPMENT – DESIGN NOT ERROR TOLERANT System design does not provide adequate redundancy to counteract errors or alerting of the effects of errors • 44 FLIGHTCREW – Flight crew failure to recognize and correct unstable approach • 100 REGULATORS – INSUFFICIENT AIR CARRIER OVERSIGHT. Insufficient regulatory oversight of air carrier operations including management and training practices

Identify Intervention Strategies • Intervention strategies – Suggested solutions – Things to do to prevent or mitigate the problem – Etc.

Intervention Effectiveness • Power – • Confidence – • Effectiveness of a specific intervention in reducing the likelihood that a specific accident would have occurred (“Perfect World”) Confidence that this specific intervention will have the desired effect Future Global Applicability – How well the intervention can be extrapolated to apply to a world-wide fleet in the future

Effectiveness Rating Scales POWER This scale is to be used to judge the effectiveness of a specific intervention in reducing the likelihood that a specific accident would have occurred had the intervention been in place and operating as intended. (“perfect world”) 0 Not at all effective 1 2 3 4 5 6 Hardly any effect Slightly effective Moderately effective Quite effective Highly effective Completely effective CONFIDENCE This scale is to be used to define the level of confidence that you have that this specific intervention will have the desired effect. 0 Not at all confident 1 Hardly any confidence 2 Slightly confident 3 Moderately confident 4 Quite confident 5 6 Highly confident Completely confident FUTURE GLOBAL APPLICABILITY This scale is to be used to estimate how well the intervention can be extrapolated to apply to a world-wide fleet in the future. (for example: how often the situation it addresses occurs in accident scenarios; whether its impact is on present and future operations (equippage, traffic, regulatory differences); and whether it is applicable across airlines/airplanes/regions. 0 Not at all applicable 1 Hardly any applicable 2 3 Slightly applicable Moderately applicable 4 Quite applicable 5 Highly applicable 6 Completely applicable

JSAT Reports • Standard Problem Statements • Interventions Prioritized • Recommendations

Joint Safety Implementation Teams (JSIT) Process

JSIT Feasibility Scales • • • Technical Financial Operational Schedule Regulatory Sociological

JSIT Safety Enhancements • Develop Safety Enhancements from Interventions • • Collect detailed resource information Prepare Detailed Implementation Plans (DIP’s)

Joint Implementation Measurement Data Analysis Team (JIMDAT) Process

Develops a Prioritization Methodology (JIMDAT) • Identifies the most effective solutions derived from all accident categories • Considers effectiveness vs. resources • Tests solutions against fatal and hull loss accidents • Creates draft master strategic safety plan • Identifies areas for future study/mitigation

Resource Cost Vs. Risk Reduction 10000 Risk Reduction $ APPROVED PLAN 9000 Total Cost in $ (Millions) 8000 75% 7000 6000 $ 50% 2007 2020 5000 4000 3000 25% 2000 0% $ Completed $ 1000 $ 0 Completed + Plan (2007 Implementation Level) Completed + Plan (2020 Implementation Level) All JSIT Proposed Enhancements (2020 Implementation Level) Resource Cost ($ Millions) Risk Eliminated by Safety Enhancements 100%

Cost Savings Part 121 Aviation Industry Cost Due to Fatal/Hull Loss Accidents 100 Historical cost of accidents per flight cycle 74 Dollars/Flt. Cycle 80 % Ri sk re du c tio n 60 Savings ~ $74/Flight Cycle Or ~ $814 Million Dollars/Year 40 Cost of accident fatalities following implementation of the CAST plan @ 2020 levels 20 0 2007 2020

General Methodology for Calculating the Potential Benefit of a Safety Enhancing Intervention Accident Risk Reduction = ( Effectiveness that an intervention has for reducing the accident rate if incorporated , Portion of world fleet with intervention implemented )

Spreadsheet Example – Historical Airplane Accidents & Proposed Safety Enhancements

Basics of the Selection Spreadsheet • Effectiveness Each safety enhancements is evaluated against each undesired condition in the set to determine how effective the enhancement would be at eliminating these conditions if the enhancement were put in place. • Implementation level is based on the portion of the affected population with the enhancement incorporated or predicted to be incorporated by a future date. • Severity Weighting To account for differences in severity or significance of the undesired conditions, a weighting value can be entered so that the relative risk of the undesired conditions is realized.

Severity Weighting Overview • To account for differences in fatality risk associated with each accident in the data set, a severity value was applied. In this assessment, the severity value represented the portion of people onboard that perished in the given accident. • Example: Comparison of two fatal accidents 757 CFIT accident, 98% perished. Weighting factor is. 98 747 Turbulence accident, . 6% perished. Weighting factor is. 006 • Hypothetically assume an assessment showed that the chance of these accident occurring would have been reduced by proposed safety enhancements by 50%. • The associated portion of fatality risk eliminated can be determined using the severity weighting factor as follows: 757 CFIT. 98 x. 5 =. 49 747 Turbulence, . 006 x. 5 =. 003

Analysis Tool Output • The spreadsheet output can be set up to show the effect that an individual safety enhancement, or group of safety enhancements have on reducing exposure to the undesired condition. Fatality Risk Reduction SE 1 SE 2 SE 3 SE 1 & SE 2 SE 1 & SE 3

Example Scatter Chart 2007 Implementation & Resources 25 20 Dollars In Millions 15 10 Combined Score 5 Combined SOPs 0 0. 0 2. 0 4. 0 6. 0 8. 0 10. 0 Score 12. 0 14. 0 16. 0 18. 0 20. 0

Safety Plan Development Accident JSITS Case studies Accident JSATS Case studies Incident Analysis Process Master Contributing Factors Emerging Risk Safety Enhancements Develop Enhancements & Metrics Recommended Plan Revision JIMDAT Review Changing Risk CAST Plan Metrics Performance To Plan Review Aviation System Changes Identify Hazards Identify Factors Yes Present In Master Factors FAST Hazards Demographic Changes Non. Performance Information Identify Hazards Identify Factors No Develop Contributing Factors (new or emerging

Robust CAST Methodology • Detailed event sequence - problem identification from worldwide accidents and incidents • Broad-based teams (45 -50 specialists /team) • Over 450 problem statements (contributing factors) • Over 900 interventions proposed • Analyzed for effectiveness and synergy

CAST Process Led to Integrated Strategic Safety Plan • Part 121 or equivalent passenger and cargo operations studied • Current CAST plan: • 72 Prioritized Safety Enhancements • 50 Complete and 22 underway • Projected 74% fatality risk reduction by 2020 • Industry and Government implementing plan

CAST Safety Plan 50 Completed Safety Enhancements • • • Safety Culture Maintenance Procedures Flight Crew Training Air Traffic Controller Training Uncontained Engine Failures Terrain avoidance warning system (TAWS) Standard Operating Procedures Precision Approaches Minimum Safe Altitude Warning (MSAW) Systems Proactive Safety Programs (FOQA + ASAP)

CAST Safety Plan (cont. ) 22 Committed Safety Enhancements • • • Policies and Procedures Aircraft Design Flight Crew Training (additional aspects) Runway Incursion Prevention Precision Approaches (additional projects) Icing (additional turboprop projects) Midair Maintenance Runway Safety culture, policies and procedures

Fatal Accident Rate and Full Airplane Loss Equivalents Rate for Part 121 Operations (5 -Year Rolling Average) 8. 0 5 year moving avg of fatal accidents per 10 million departures 7. 0 6. 0 5. 0 4. 0 3. 0 82% Fatality Accident Rate Reduction 1996 -2007 2. 0 1. 0 7 20 0 6 20 0 5 20 0 4 20 0 3 20 0 2 20 0 1 20 0 0 20 0 9 19 9 8 19 9 7 19 9 6 19 9 5 19 9 4 19 9 3 19 9 2 19 9 1 0. 0 19 9 Fatality Accidents or Full Loss Accident Equivalents per 10 Million Departures 9. 0

International Perspective CAST Safety Enhancements United States and Canada CAST 72 SEs 50 complete 22 in work ECAST 35 SEs JAA 11 SEs EASA Europe C. I. S. 1 JAA Non JAA COSCAP CIS Specific CIS Projects Middle East COSCAP GS COSCAP BAG Africa COSCAP CEMEC Latin America and Caribbean RASG-PA COSCAP UEMOA China Asia (Excluding China) COSCAP NA, SEA 40 SEs in work Oceania

CAST Phase 2 Transition to Decision Making Based on Analysis of Incident and System Safety Performance Data Aviation Safety Information Analysis Sharing (ASIAS)

The Aviation Safety Information Analysis and Sharing (ASIAS) System Was Created in Order To: • • • Develop tools to make data analysis more efficient • Develop automated information integration capabilities centered on aviation safety risk topics • Transfer technologies and key data sources into National Archives Identify and access key data sources Discover potential aviation safety risks using the key data sources

ASIAS Enables Various Types of Proactive Safety Analyses Known Risk Monitoring Vulnerability Discovery Benchmarking Operations Safety Enhancement Assessments Directed Studies

ASIAS Has Access to Multiple Data Sources Airline Data FAA Other Sources • Digital Flight Data • Voluntary Pilot Safety Reports • Surveillance Data • Traffic Management Reroutes & Delays • Airport Configuration and Operations • Sector and Route Structure • Procedures • Safety Reports • Bureau of Transportation Statistics • Aviation Safety Reporting System • Weather / Winds • Manufacturer data • Worldwide Accident Data

ASIAS is Governed by Formal Principles Data used solely for advancement of safety Non-punitive reporting Airline data is de-identified Analyses approved by an ASIAS Executive Board

Airlines Participating in ASIAS 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. Air. Tran Airways Inc Alaska Airlines American Eagle Atlantic Southeast Airlines Chautauqua Airlines Compass Continental Delta Airlines Express Jet Frontier Airlines Gulfstream Intl Jet Blue Northwest Republic Airlines Shuttle America Sky. West Southwest United UPS US Airways • 21 ASIAS airlines with signed Memoranda of Understanding (MOUs) (as of July 7, 2009) • 75% of 2008 operations

Participating Airlines 21 airlines participating in ASIAS representing 75% of 2008 operations (as of July 7, 2009) size of circle: 240, 000 ops 44

Benefits of ASIAS With ASIAS, the aviation community will be able to. . . • Identify Systemic Risks • Establish safety baselines of current operations • Identify known and newly emerging system • • • vulnerabilities • Monitor safety trends Evaluate Identified Risks • Estimate their probabilities • Assess their severities • Uncover event precursors • Diagnose event causation Formulate Interventions • Assess the probable effects of safety enhancements through simulation studies Monitor Intervention Effects • Assess the effectiveness of interventions (SEs) in accordance with metrics established by the CAST

ASIAS Studies In Progress or Completed Runway Safety Directed Studies Terrain Awareness Warning System Study TCAS Resolution Advisories CAST Known Risk and Safety Enhancement Effectiveness Monitoring Risk of Landing Runway Overrun Approach and Landing Accident Risks Controlled Flight Into Terrain (CFIT) Terrain Awareness Warning System Alerts Airline Benchmarks Unstabilized Approaches TCAS Resolution Advisories

Prioritization Issues/Considerations • There is a need for CAST to prioritize its desired tasks so that MITRE resources and CAST Subject Matter Expert (SME) support can be managed effectively • Use of MITRE resources and associated CAST SME support needs to be looked at in total and each task given a priority 47

CAST Use of Data Status: currently ongoing: Potential safety concerns are being Developed & prioritized by JIMDAT. CAST will request AEB to allocate MITRE resources to build query tools and provide query results Study items from outside requests Processed data Data (ASIAS and any other data) Data Queries & Data Processing Threshold Filter Attention Items (Disseminate information to CAST when above a TBD threshold) Contributing Factors Understood? No Meet Study Guidelines? Yes Prioritize CAST Study Items (Risk/Resources) Yes No CAST may direct that additional metrics or trend monitoring be developed (May require AEB approval) No Perform study on selected item (May require AEB approval – data & MITRE resources) Is risk level sufficient for mitigating action? Yes Develop SE Metrics per CAST direction (May require AEB approval) Initiate CAST SE Development 48

ASIAS Data Queries & Data Processing Based on issue priority, CAST will request resources (AEB/MITRE, SMEs) for tool development, data queries, data processing. A Threshold B C Potential Study Item CAST Precursors and Issues Thresholds, exceedences, and abnormalities will be unique to each issue and have yet to be established 49

Safety Study Decision Guidelines 1. AIRPLANE ACCIDENT SAFETY YES NO Supporting data A. The condition of interest was likely a significant contributing factor in a Part 121 or equivalent catastrophic event (including relevant events on other classes of operations). B. The condition of interest is considered to have a significant adverse influence on the effectiveness or implementation of an approved safety enhancement that was not considered in the SEs original assessment C. 1. 1. 1) The condition of interest is considered to be a significant contributing factor to the occurrence of a catastrophic accident in the future and: there is an expectation of at least one catastrophic accident within the next 20 years. OR 1. 2) the probability* of one or more catastrophic accidents is greater than 4 x 10 -6/flt cyc. *Average per-cycle probability as applicable. 2. PERSONAL SAFETY The condition could result in serious injury or death to crew or passengers, and cannot be shown to be less probable than 1 E-5/flight cycle (TBD). 3. OTHER An event/condition that any CAST member considers to be a condition of study provided rationale is presented and those resources needed for the study are approved by the affected CAST members. 50

Risk Level Appropriate for Monitoring 1 accident in 20 yrs Risk Level Appropriate for Study RR SEs Risk Levels nse o esp id R l– k Ris ve Le p Ra Fatality Accident Expectation @ Current Accident Rate 51

Summary • History shows focused action and introduction of new capabilities have led to accident risk reductions • Joint industry and government teams working together to a common goal can further enhance the safety of our very safe aviation system • CAST has detailed plans in place to address known problems • CAST is refining incident analysis and operational data mining to uncover problems before they result in accidents 52

http: //www. cast-safety. org/