International Civil Aviation Organization COG Performance Task Force

International Civil Aviation Organization COG Performance Task Force Workshop Rome 21 -22 September 2011 EASA Standardisation Inspections Verification of Safety KPIs Milen Dentchev, ATM/ANS Standardisation Manager ICAO EUR/NAT Office COG PERF TF Workshop 1

EASA Standardisation – Legal Basis Where Member States act as Competent Authority the EASA Basic Regulation establishes a mechanism to monitor the correct implementation of applicable regulations. QEASA Basic Regulation (EC) No. 216/2008: - Art. 24 (Monitoring the application of rules) - Art. 54 (Inspections of Member States) QWorking methods regulated by Commission Regulation (EC) No 736/2006 2 ICAO EUR/NAT Office COG PERF TF Workshop 2

Organisational Structure Head of Standardisation T. Mickler Deputy Ho. D M. Kompare Secretariat Development C. Trevisan Airworthiness M. Kieft IAW CAW Accreditation J. Müller ACC Air Operations G. Rebender CF NCF Flight Crew Licensing G. De Crescenzo LIS FSTD MED ANS M. Dentchev ATM CNS AIS MET LIS OCC 3 ICAO EUR/NAT Office COG PERF TF Workshop 3

Standardisation Department (S. 1) • Tasks - Standardisation (“EASA Countries”) - Standardisation (“ECAA” and “WA” Countries) - International Standardisation (“Third Countries”) - Accreditation (NAAs working for EASA) ICAO EUR/NAT Office COG PERF TF Workshop 4

Geographical Scope IS The playing field: 27 EU 3 EEA 1 CH 7 ECAA 8 ECAC FL SE NO RU EE LV DK RU IR UK BE UA DE CZ LU SK CH AU LI RSM MC BA RO RS ME IT ES MD HU SL HR PT KZ BY PL NL FR LT KO FY AL GR GE ICAO EUR/NAT Office DZ TN MT COG PERF TF Workshop AZ BG IR TR SY MA AM IQ CY 5

EASA Inspection process Preliminary meeting Opening session Closing session Wrap-up meeting Finding Classification Committee On Site 10 weeks min As necessary 12 weeks max 16 weeks max Preparatory phase Visiting phase Reporting phase Follow-up phase Notice of inspection Visit programme Preliminary Report Final Report As necessary Closure Phase Action Plan Closure Supplementary Report ICAO EUR/NAT Office COG PERF TF Workshop 6

Standardisation Strategy Successful standardisation cannot depend only on regulatory compliance verification – Performance verification is part of regulatory verification EASA’s strategy is based on three pillars: Standardisation Strategy Regulatory Compliance verification Regulatory feedback Pro-active measures ICAO EUR/NAT Office COG PERF TF Workshop 7

NSA Standardisation Forums NAA Partnership meetings ANS National Coordination meetings Post ATM/ANS Standardisation visit meetings Internal EASA FCC meetings ICAO EUR/NAT Office COG PERF TF Workshop SES NCP NSA Standardisation Coordination meetings 8

Increase added value Mo. U between EU and ICAO – Addressing aviation safety, aviation security, air traffic management and environmental protection – New Working arrangements – Transparency through mutual exchange of safety information and access to databases – Reduce burden of multiple audits/inspections on Member States ICAO EUR/NAT Office COG PERF TF Workshop 9 9

Second extension of EASA Regulation (EC) No 1108/2009 amending EASA Basic Regulation – EASA mandated to conduct ATM/ANS standardisation inspections – Regulation 736/2006 will be amended accordingly – Transition from EUROCONTROL ESIMS audits to EASA Standardisation inspections ICAO EUR/NAT Office COG PERF TF Workshop 10 10

Scope of ANS according to ICAO Annex Annex Annex ICAO EUR/NAT Office 2 - Rules of the Air 3 - Meteorology 4 - Charting 5 - Units of measurements 10 - Aeronautical telecommunications 11 - Air Traffic Services 12 - Search and Rescue 13 - Accident & Incident Investigation 15 - Aeronautical Information Services 16 – Environmental protection PANEP meeting 17 -18 May 2011 COG PERF TF Workshop 1 11 1

Scope of EASA ANS standardisation inspections Annex Annex Annex ICAO EUR/NAT Office 1 - Personnel Licensing (ATCO + ATSEP) 2 - Rules of the Air 3 - Meteorology 4 - Charting 5 - Units of measurements 10 - Aeronautical telecommunications 11 - Air Traffic Services 12 - Search and Rescue 13 - Accident & Incident Investigation (SOR) 15 - Aeronautical Information Services 16 – Environmental protection PANEP meeting 17 -18 May 2011 COG PERF TF Workshop 1 12 2

Transition from ESIMS to EASA inspections • EASA’s ATM/ANS standardisation section: • Checklists and pre-visit questionnaires in preparation • Standardisation inspection annual programme ready • NSAs invited to confirm or nominate National Coordinator • NSAs have been invited to identify and nominate qualified inspectors to participate in EASA standardisation inspections • 1 st training course for NAA inspectors in Jul, 2 nd in Sep, next in Jan 2012 (40 people trained) • 6 EASA Team Leaders Q 1 of 2012, 9 Team Leaders Q 2 ICAO EUR/NAT Office COG PERF TF Workshop 13

Amendment of WA At the request of the European Commission, EASA has approached all WA countries (including all non EASA – EUROCONTROL countries) with a proposal to extend existing WAs to ATM/ANS. All ECAA States responded positively Non-ECAA States: Positive answers from Ukraine, Armenia, Georgia, Moldova, Monaco & San Marino No answer from Azerbaijan yet Turkey – no decision yet ICAO EUR/NAT Office COG PERF TF Workshop 14 14

Verification of safety performance KPIs ICAO EUR/NAT Office COG PERF TF Workshop 15 15

Confidence Level: the concept Activity Indicators Safety occ Indicators ADDITIONAL FACTORS Authority NAAIndicators CORE Follow-up INDICATORS Indicators Time Elapsed 08. 2011 ICAO EUR/NAT Office General Indicators COG PERF TF Workshop = Confidence Level 16

Core indicators: the NAA and its follow-up The model evaluates firstly two «core indicators» : Authority Complexity (structure, ROs, . . . ) Funding / Salaries Staffing Stability Maturity Staff qualification and training Internal guidance External guidance Oversight methodology Management of findings Enforcement capabilities Reaction to AIBs' rec’s ICAO USOAP/CMA results Reactivity Index Supplementary Reports Repetitive key findings This is, to a large extent, the current scope of our standardisation activities Follow-up 08. 2011 ICAO EUR/NAT Office COG PERF TF Workshop 17 17

Additional indicators The model then takes into account the following indicators: Activity Safety Aircraft registered Organisations approved Personnel licences issued Traffic indicators Domain-specific indicators Accidents SAFA Country ratio Operators in "high SAFA“ list Immediate Safety Hazards Domain-specific indicators 08. 2011 ICAO EUR/NAT Office COG PERF TF Workshop General NAA Independence Safety culture Reporting culture Just culture Corruption index Financial indicators Major gov’t changes 18 18

The influence of time Finally, the model considers the influence of time elapsed since the last inspection, as the reliability of the information on the NAA’s capability and performance taken at that stage will inevitably decrease with time: De cr = Time elapsed Confidence level ea se wit ht im e Threshold Time elapsed 08. 2011 ICAO EUR/NAT Office COG PERF TF Workshop Inspection due! 19 19

Vulnerability Index - Confidence levels (CLs) are determined for each domain: - The lowest CL would highlight the «most vulnerable» area - A Vulnerability Index may therefore be defined as follows: VI = 100 – min (CLs) 08. 2011 ICAO EUR/NAT Office Note: work in progress! COG PERF TF Workshop 20 20