POST AUDIT ACTIVITIES 1 Audit Report Writing Transmission

POST AUDIT ACTIVITIES 1 Audit Report Writing, Transmission and Follow-up

2 Module Objectives By the end of the module the Trainee should be able to: q Review inspection outcomes q Distinguish between inspection/audit findings and observations q Write an inspection report q Fill Corrective Action Request forms q Transmit an audit report q Evaluate the ANSPs’ Corrective Action Plans q Follow up on CAP implementation

3 What is a Report A report is a document that presents information in an organized format for a specific audience and purpose. It gives an account of something that one has observed, heard, done, or investigated. The report is meant to achieve an organizational purpose of: q Informing q Putting in record, and/or q Requiring action

4 Post Audit Activities After physical or virtual surveillance, audit teams conduct a post-audit meeting to consolidate and analyze data. Once conclusions have been drawn, audit teams are expected to conduct a validation meeting with the Head of the inspectorate and finally to complete and submit an audit report to the ANSP(s). Post audit activities include: collating and analyzing data collected from all audited facilities and services; validating observations; writing and submitting an audit report; archiving audit records; and follow-up with the ANSP(s).

5 Audit Report A document that summarizes the results of an audit It gives a factual account of the audit Highlights observations made and if applicable recommendations Prepared by the team leader in coordination with team members Should be approved by the head of inspectorate (as applicable) before submission to ANSP

6 Draft Audit Report Serves as a post-audit brief detailing findings and recommendations Also serves as an advance notification for the ANSP to submit Corrective Action Plans Used to verify findings/observations made during the exit meeting

7 Audit Report The final audit report is an objective reflection of the audit It details the audit process and summary of identified deficiencies Written to formalize audit findings and request for corrective action by the ANSP Should be clear, concise and contain facts presented and agreed upon at the exit meeting

8 Audit Report Writing Commences immediately after the exit meeting. The Team Leader (TL) consolidates contributions from Team Members. The TL should review, coordinate and discuss with Team members of their individual submissions. This ensures the overall quality and consistency of the report.

9 Audit Report An audit report should objective and not contain: Subjective opinions Criticism Observation not discussed at the exit meeting Confidential information Trivial issues

10 Audit Report As part of the production of the audit draft report, the TL performs a technical review of the audit results and the information contained in the audit draft report, including the: q evidence collected by the audit team; and q findings issued by the audit team. q recommendations of the TMs

11 Findings Generated as a result of an audit to point to non-compliance with safety related provisions of: q state regulations q ICAO annexes (where applicable) q Technical Guidance Material q Good aviation safety practices q Non development of a required document/manual

12 Classification/Level of Findings are classified according to their possible effect on air safety Inspectors should therefore evaluate the effect of each finding on safety Deficiencies posing imminent safety risk must receive immediate attention If safety is/will be compromised by continued existence of a deficiency, then action should be taken to suspend operations

13 Classification of Findings can be classified in three classes namely: Serious Finding (Level I) Major Deficiency (Level II) Minor Deficiency (Level III)

14 Serious Finding (Level I) A deficiency posing a very serious safety risk to aviation safety Requires immediate action by the ANSP to resolve it Involves imposition of operating restrictions till the safety concern has been addressed or resolved No certificate/licence or authorization should be issued or renewed till the deficiency is resolved or eliminated.

15 Major Deficiency (Level II) A deficiency in the service provider’s system and posing a serious safety risk which should be resolved within 10 days or such other period determined by the State Requires a CAP from the ANSP detailing plans and timelines to resolve the deficiency The Authority can impose operating restrictions to ensure the ANSP resolves the deficiency Certificate/licence/authorization action till an acceptable CAP is received is an example of operating restrictions

16 Minor Deficiency (Level III) A non-compliance that could lower safety standards Poses less serious safety risk and concerns isolated incidents which will not necessarily have a direct impact on safety on its own Required development of a CAP by the ANSP with a maximum implementation period of 90 days or such other period determined by the State A certificate/licence/authorization may be issued/renewed subject to submission of an acceptable CAP by the ANSP

17 Minor Deficiency (Level III) cont. The Air Navigation Service provider shall inform the CAA in writing on completion of corrective actions for all deficiencies identified. The ANSP can apply for exemption while providing suitable mitigation measures where the deficiency cannot be resolved within 90 days or any other period determined by the State Exemptions shall be granted for a period not exceeding that specified in the regulations

18 Parallel Findings against the CAA Involves deficiencies in or misapplication of Civil Aviation legislation, policies and procedures Findings against the CAA will be described in a parallel report The parallel report is forwarded to the CAA by the audit manager with notes identifying the problem, cause, responsibility and recommended solutions. The CAA deficiencies shall neither be included nor referenced in the audit report.

Parallel Finding Follow-Up 19 Parallel finding items shall be forwarded to Head of Oversight, who will assign an appropriate office for coordination and follow-up of those deficiencies The CAA is responsible for resolving these inconsistencies and advising the auditee of any required action

20 Parallel Observations (System Weakness) These observation findings will be documented on the audit finding form as an observation The purpose of parallel observations is to improve the quality and effectiveness of the CAA regulatory oversight program

21 Observations May be negative or positive They must not include information suggesting noncompliance with Civil Aviation Regulations and/or procedure No regulatory action is required to be taken in the case of observation; Service providers are however urged to take observations seriously for purposes of improving their operations.

Audit Report Format 22 1. Executive Summary Main message: a few sentences summing up your findings, conclusion and recommendation. 2. Introduction Background What you inspected and why (objective); who was involved; the scope (where); when; method of inspection 3. Main Body Contains the details of the audited/inspected areas including observations and findings? 4. Conclusion (Outcome) Contains results, evaluations and what action should be taken on the findings identified during the audit/inspection. 5. Recommendations What should be done to meet required standards or continued compliance. 6. Appendices/Attachments Supporting materials. e. g. checklist, Corrective Action Request (CAR) form, parallel finding form etc.

Audit Report Template 23 XXX CIVIL AVIATION AUTHORITY AUDIT REPORT File Reference Name of ANSP: Area Audited: Date of Audit: Auditor(s) Audit Summary Audit Findings: Recommendations Prepared By: Date:

24 Attachments to an Audit Report The completed report shall have the following attachments as applicable: q Completed Checklists q Corrective Action Request form(s) q Observations forms q Parallel finding form q Any other necessary documents

25 Audit/inspection report submission Once the team leader has assembled the global report, it will be forwarded to the head of inspectorate for approval. Timelines specified by the CAA will be followed in submission of the final report

26 Corrective Action Requests (CAR) The audit report transmitted to the audited entity will be accompanied with individual Corrective Action Requests (CAR) for each finding identified during the audit/inspection

Corrective Action Request Form CORRECTIVE ACTION REQUEST (CAR) 27 File Reference: Level of Finding: Name of Operator/Service Provider: □ Level II Date of Audit: □ Level III Area Audited: Applicable Regulatory Requirement Reference: Description of Finding/deficiency: Corrective Action Plan to be submitted within …… days from the Date of this CAR (the time frame shall be determined by the Level of finding/deficiency) Name of CAA Inspector: Signature: Date:

28 Corrective Action Plan (CAP) The ANSP is required to develop acceptable corrective action plans (CAPs) for each finding The CAPs should be submitted to the CAA for evaluation within the specified timelines Corrective actions to be taken by the ANSP shall be determined in accordance with the levels of the respective findings

Sample CAP Form 29 OPERATOR/ SERVICE PROVIDER’S CORRECTIVE ACTION PLAN Name of Operator/ Service Provider’s: Location: Date: File Ref: Details of Finding: Immediate or Short Term Corrective Action: Completion Date: Long Term Corrective Action: 1. Cause(s) of Problem 1. Action taken / to be taken Proposed Completion Date: Operator/ Service Provider’s Representative Name: Signature: Date:

30 1. Short-term corrective action A preliminary action developed by the ANSP to correct a deficiency Short-term corrective action will be completed by the date/time specified in the corrective action section of the CAP form, or as per the accepted corrective action plan

31 2. Long-term corrective action Has two components The first component will involve identifying the root cause of the problem and indicating the measures the ANSP will take to prevent a recurrence The second component is a timetable for implementation of the long-term corrective action Must have a proposed completion date

32 CAP steps and proposed action items Ensure that the proposed actions in a CAP directly and fully address the finding. If required, break down large action items into smaller and more manageable steps. Describe each proposed action in a clear and detailed manner. List the step-by-step corrective actions in the correct sequential and/or chronological order (e. g. establishing a requirement before implementing it).

33 CAP steps and proposed action items Provide a good and clear working plan and adequate detail for the implementation of each proposed action Provide necessary details on implementing requirements and procedures

34 Action office Ensure that the responsible action office is indicated for each one of the corrective action steps. If more than one organization or entity are involved in each step, identify and record each one clearly. Ensure that the action offices identified in each step of the corrective action have the authority to complete the action, especially with respect to the promulgation of legislation and/or regulations.

35 Action office For higher-level corrective actions, the ANSP should enter the name of the entity which has the authority to complete the action. Spell out the acronym for the title of an action office the first time it is used in the CAP; use the acronym thereafter

36 Evidence reference When referenced evidence is contained in a document, an indication of the document containing the evidence should be made in a clear manner. Specific and clear reference to the page, section or paragraph of the document that contains the information should be provided so that the evaluating inspector can easily review and validate the CAP. Broad and generic reference to a large document should be avoided. Be as specific as possible.

37 Estimated implementation date Ensure that an estimated implementation date (est. imp. date or EID) is entered for each step. Ensure that the EID is realistic for the action item. Ensure that the EID is appropriate for the level of risk associated with the finding. The EID should be the date of completion for the action item

38 Submission of CAPs The service provider shall submit CAPs to the CAA as per the guidance given by the Authority. Specified timelines will be followed CAPs received from the ANSP should include completed CAP forms and where applicable, supporting documents that may take the form of technical record entries, purchase orders, memoranda, revised inspection/audit procedure cards, manual amendments etc.

39 Evaluation of Corrective Action Plans CAA inspectors evaluate CAPs for their ability to resolve identified deficiencies For deficiencies that pose imminent safety risks, there may be no time for providing and accepting a written CAP. In such cases the operator/service provider may proceed to implement corrective action and notify the Authority of the action taken.

Sample CAP Evaluation Form 40 CORRECTIVE ACTION PLAN EVALUATION & TRACKING CAA RESPONSE/COMMENTS: □ CAP Accepted Proposed Follow-Up: □ On-site □ Administrative Proposed Follow-Up Date: □ CAP Rejected New CAP Target Date: Inspectors Name: Inspectors Signature: Date: Finding Closed By: Date of closure: □ Yes Name: Signature

41 Acceptance of CAPs An acceptable CAP must have: q a defined action; q an implementation timeline that is reasonable and commensurate to the level of deficiency; and q a definite action office responsible for its implementation.

42 Acceptance of CAPs not meeting the requirement will be rejected and the ANSP so informed. The ANSP shall be required to provide a new CAP. Failure by a service provider to submit an acceptable CAP may result in the deficiency being referred for enforcement action and/or imposition of sanctions. Where the CAP is acceptable the ANSP will be informed as such.

43 Corrective Action Plan follow-up Accepted CAPs should be implemented within the stipulated timelines The CAA shall keep track of all CAPs submitted by service providers and follow up on their implementation In preparation for surveillance activity and/or ad-hoc audits, inspectors shall review all pending CAPs for the concerned ANSP and assess their level of implementation.

44 Follow-up process Where the inspection/audit findings are of a minor nature and no direct threat to aviation safety exists an “administrative follow-up” may be acceptable All other findings require “on-site follow-up” Progress will be monitored as the service provider completes inspection/audit finding corrective actions Long-term corrective actions will be followed-up through routine surveillance activity.

45 Tracking of CAPs and Audit/ inspection Closure Follow-up responsibilities include: monitoring the service provider to ensure that the appropriate response time for corrective action plan submission is observed ensuring that CAPs addressing the most critical findings are prioritized ensuring that each proposed corrective action will rectify the root cause of the finding to prevent its recurrence

46 Tracking of CAPs and Audit/ inspection Closure determining that the service provider has developed a reasonable timetable for long-term corrective action accepting the CAP in co-ordination with the head of inspectorate and appropriate team leader and/or team member determining for each corrective action plan item whether the follow-up is to be administrative or on-site

47 Tracking of CAPs and Audit/ inspection Closure monitoring the progress of the corrective action plan by maintaining the follow-up section of the corrective action form. ensuring that all completed CAP forms and corrective action tracking forms, together with any supporting documentation are stored and archived as necessary. advising the head of inspectorate when all corrective actions have been completed.

48 Updating CAPs CAA inspectors should ensure that CAPs are continuously updated by indicating the: progress level for each action item as it is implemented; and the date of completion for each completed action item. If the initial EID of an action item has passed and the action has not completed yet, the ANSP should provide a new EID.

Sample CAP Follow-up & Closure Form 49 No. Date of Inspector Details of findings Inspection (s) 5 th June 2019 Grand Cregg JAAP/ATM/01/06/2019: The ATS provider has a shortage of rated and valid ATCOs leading to long working hours in contravention of the CA (PEL) regulations. Details of Corrective Action Plan The ATS providers management has commenced a recruitment and training exercise to attain required controller numbers. Status of finding (closed/open) Open Remarks Cause: Government bureaucracies CAP date: 30/09/19 Recruited cadets issued with letters to report at the ATC training center as at 7. 9. 2019 New CAP date 30. 6. 2020 11 -12 th Sept. 2019 Grand Cregg Alex Abbey (OJT) Parry Joe (OJT) JAAP/ATM/02/09/2019: The tower cabin is subjected to high noise levels due to proximity with aircraft operating on the new apron and the lack of sound proofing mechanisms in the tower cabin. 1. Each Controller Open has been provided with headsets to mitigate the effect of noise on communication. 2. Measures to be taken to enhance insulation of the TWR cabin to limit noise penetrating its Action 1. completed on 20. 11. 2019. action 2. has a CAP completion date of 30. 3. 2020

End of Chapter 50 Thank You