Introduction to After Action Review online course Version

Introduction to After Action Review (online course) Version: 17/09/18

Learning Objectives E- Learning Course 45 minutes Introduction to After Action Reviews At the end of this module, you will be able to: • Understand the principles of After Action Reviews • Explain the working group After Action Review (AAR) process • Understand the role of facilitators and note takers in the process and describe participatory facilitation techniques • Recognize the key factors for a successful working group AAR

Should I take this elearning course? When should I take this e- learning?

Course Content 1. Description and purpose of After Action Reviews 2. Key steps of After Action Reviews 3. Facilitator preparation and guidance

What is an After Action Review? An After Action Review (AAR) is a qualitative review of actions taken to respond to an emergency as a way of identifying best practices, the gaps and lessons learned It is a space for collective learning by bringing together the relevant individuals to critically and systematically analyse actions taken during the response

How does it fit within the IHR Monitoring & Evaluation Framework? Assessments conducted by Mo. H or jointly with WHO under the IHR monitoring and evaluation framework (IHRMEF) • Joint External Evaluation: Desk review conducted by external experts of 19 JEE technical areas using the JEE tool AR AAR Simex Real country capacity • Annual Reporting: National selfevaluation of capacities submitted on a yearly basis covering 13 technical areas (2017) • After Action Review: Qualitative review of actions taken to respond to a real emergency for the purposing of learning and assessing the functionality of capacity • Simulation exercises to develop, assess and test emergency systems, procedures and mechanisms to respond to outbreak and public health emergencies.

AAR as part of preparedness and response cycle Identificati IHR National on of gaps After Action Plan and best Action practices IHR Review capacity Simulation building in Exercise the Organising Emergenc emergency systems y response and Training cycle building and capacity equipping Annual reporting JEE (every 5 years)

LEARNING ATTRIBUTES OF AAR Personal learning Learning within a team, for better team dynamics and understanding within emergency management process Collective learning Learning on individual level role within the management of an emergency Institutiona l learning Improvement to overall emergency management, such as through improved preparedness and response plans, mechanisms of coordination, etc

Why do an After Action Review? Assess functional capacity of existing systems to prevent, detect and respond to Public Health Event Contribute Identify to transparency and mutual accountability practical, actionable steps for improving preparedness & response systems Document, Record & Share lessons learned with other public health professionals Achieve collective vision amongst response stakeholders of how to do better in the future

Key actions carried out during AARs Objective observation Establish how actions were actually implemented during the response, in contrast to how they are supposed to or normally happen, according to plans and procedures. Analysis of gaps and best practises and contributing factors Identify the gap between planning and practice. Analyse what worked well and what worked less well and why. Identify areas of improvement Identify actions to strengthen or improve performance and how to follow up

What are the key principles of an after action review • It is participative – participants are encouraged to engage in an active and respectful manner • Open and honest feedback is given • There is space for experience sharing and mutual learning • There is analysis of systems and processes and not an evaluation of individuals • The intent is to identify solutions and compile a list of key activities/recommendations to improve future responses

4 formats of an AAR proposed by WHO Debrief Working group Key informant interview Mixed method Factors affecting the format selection include location and number of participants, cultural context, health event complexity and resources required. More info in the AAR Guide (link)

How should AARs be organized? Debrief Working group Key informant interview Mixed method Debrief (Narrow scope): • Small group discussion • Informal Deeper analysis of the technical area and maximize learning • focus on a specific technical area • involve only individuals and partners that were involved in this area • should not take more than half a day max.

How should AARs be organized? Debrief Working group Key informant interview Mixed method Working group AAR (medium scope): • Larger, more diverse, group of participants • cover multiple technical areas of a response in parallel • Done through structured working group discussion • usually takes 3 days highlight synergies between areas & stakeholders of a response

How should AARs be organized? Debrief Working group Key informant interview Mixed method Key informant interview: • one-to-one interviews with key stakeholders • Appropriate where responders are no longer in the same place or if complexity of the event requires it. • But do not benefit from the value of collective analysis • Less ownership over results Allow to express ideas & opinions more freely than in group formats

How should AARs be organized? Debrief Working group Key informant interview Mixed method (comprehensive review): • Suitable for full-scale AARs involving a larger number of participants and stakeholders • This can be a combination key informant interviews, and debrief or working group AAR formats • This can be resource and time intense • This requires an AAR team to consolidate and document results Mixed method in-depth analysis including the decisionmaking and strategic levels

Toolkit for the 4 methods Debrief Working group Key informant interview Mixed method Examples of Tools: • Templates of concept notes, agendas, note takers forms, reports, evaluations etc. • Planning checklist and template budgets • Facilitator guidance manuals and supporting PPT presentations More details about the 4 AAR formats can be found at this links: • AAR guide (link) • AAR toolkits (link) The AAR Toolkit contains guidance and tools for all 4 methods

Introduction to working group format AAR This e-learning focuses on the facilitation of working group AAR format AAR, for being the most implemented AAR so far. This AAR format: • involves the analysis of multiple functions of a response such as surveillance, coordination or communications (these will be determined prior to the AAR) • brings together 20 -50 participants of the relevant stakeholders in the response • takes 2. 5 to 3 days • is used in instances where those involved in a response can be brought together to share learning and experiences in an open and honest environment.

Stages of the After Action Review process This section of the e-learning describes the steps of a group work After Action Review. The 5 sessions follow a structured methodology with user-friendly material, group exercises and interactive facilitation techniques. Session 1 – What was in place before the response? Session 2 – What happened during the response? Session 3 – What went well? What went less well? Why? Session 4 – What can we do to improve for next time? Session 5 – The way forward

Session 1 – What was in place before the response? Objective : to establish what was in place prior to the response that should have been used to support the response

Session 2 – What happened during the response? Objective: To discuss and agree upon the key events of the response and build a comprehensive timeline

Session 2 – What happened during the response? Example of timeline of key events during the Pulmonic Plague in Madagascar in 2017 (AAR conducted in july 2018)

Session 3 – What went well? What went less well? Why? Objective: To identify the key challenges and best practices encountered during the response, their impacts on the response, and the factors that lead to them Best Practice Impact(s) Enabling factors • Improved coordination and sharing of information for the early detection of suspected / confirmed cases and for monitoring contacts • Relationship had been established prior to the response • Willingness of all stakeholders to undertake regular meetings • Political and financial support from central level Challenges Impact(s) Limiting Factors Coordination at local level ineffective • Response not coordinated between partners, health authorities and the central level • Duplication of activities and effort • Inefficient use of resources Regular cross border coordination meetings • Lack of a plan for district level coordination • Partners not participating coordination meetings

Session 3 – What went well? What went less well? Why? Example: Nigeria Lassa Fever response AAR (2018) Best Practice Impact(s) • Development of National algorithm for Lassa Fever • Improved quality of test results • Standardization of laboratory procedures • Collaboration between State, NGOs and partners to reach a larger community on Lassa Fever Awareness and sensitization • Increased coverage and awareness • Improved positive behavioural change Enabling factors • Existing collaboration with laboratory stakeholders led by the Nigeria Centre for Disease Control (NCDC) and WHO • Prior established relationship between state health educators and NGOs/partners • Involvement of NGOs and partners in planning and implementation • Regular meetings with NGOs and partners

Session 3 – What went well? What went less well? Why? Example: Nigeria Lassa Fever response AAR (2018) Challenges Impact(s) Limiting factors • Stockout of consumables and supplies for LF case management e. g. Ribavirin, Hand gloves in all treatment centres • Delay in commencing treatment for patients • Increase in mortality • Exposure of HCWs • Number of patients seen in the outbreak greater than expected • Paucity of manpower in treatment centres • Available HCWs overwhelmed with high patients workload • Increased risk of exposure of HCWs • Poor human resource for health planning

Session 4 – What can we do to improve for next time? Objective: To identify the key activities that can be undertaken in order to overcome challenges and embed best practice for future responses.

Session 5 – The way forward Objective: To clarify the way forward for activities defined through the workshop and define the final steps in the AAR process

What are the key steps for facilitators in the group AAR process? E-learning completed Read Facilitator guidance document Attend Pre AAR meeting Co-facilitate Action Review Exercise Follow up on key activites post AAR

Tick/click on? ? those things that apply to an after action review: 1. It requires participants to be open and honest in their feedback 2. It involves an external evaluation of the response 3. It is a place for sharing and learning from individuals experiences 4. It is a chance to identify what went well and what could be improved 5. It is a chance to identify individuals who did not perform well

Rules of Participation The below is the guidance given to participants. As a facilitator you are asked to ensure: l Working sessions start and end on time l Different points of view are respected l Writing is legible and clear l Participant stay engaged l Participants do not use laptops, mobile phones l Groups used the colored cards/post its assigned to them throughout the workshop

Question Take 1 minute to identify 3 attributes of a good AAR facilitator Attributes could include: l Encouraging and inclusive l Non judgmental l Allows the group to speak l Neutral - doesn’t allow their opinion to influence the group l Helps the group to stay focused on the task

Facilitator Tips l l l Maintain an unbiased perspective and use open-ended questions to guide the discussion. Focus on learning. AAR is not an evaluation of performance but an opportunity to learn challenges and best practices. Encourage people to give honest opinions. Encourage active participation by all participants, including silent participants. Encourage groups to write legibly on cards, post-its, and flipcharts.

What to avoid as a facilitator l Criticize, blame or judging the feedback l Focusing on the negative. An AAR is as much about recording and analyzing what worked well as about what did not work. l While the AAR is a learning framework, the teaching of participants should be avoided. l Do not allow your own opinion or experience to influence or disrupt the conversation of groups

Root Cause Analysis A method to identify causal factors that lead to successes or challenges in relation to identified problems. It should be used when a problem is identified that clearly requires deeper examination or for which the why this problem occurred is not answered. 5 Whys method is the simplest and most frequently used. In essence, the facilitator repeatedly asks “why” in order to progressively unpack causative factors in order get to the root cause of a particular issue. Listen to the conversation below. Consequences Problems Causes

Case study You are working with a group who has several dominant participants. You notice that several people have not contributed to the discussion. They appear to have become disengaged from the process and are no longer trying to contribute. Take a minute to think about what you might do in this situation to help ensure everyone is able to have their say? They can then click on an arrow/button that leads to this text: Ideas to encourage participation in this context could include l breaking the group into smaller groups to allow more active participation. Allow them time to work together before feeding into the group. l asking people to discuss with their neighbor their ideas before asking each pair to share l Do an individual reflection session and get participants to write their ideas on post it notes and then invite them to come and bunch them all on the flip chart

Tips for note takers l Transcribe everything written on the cards, flip charts, sticky notes during Sessions 1, 3, 4 and 5 in the provided templates. l Use the template from the lead facilitator for taking notes. This is in the same format as the sessions to enable you to record everything that is written. l When additional information is discussed but not written by the group, be sure to save it for discussion during subsequent sessions.

Click on the below steps in the AAR process in the correct order What went well? What went less well? Why? (number 3) The way forward (number 5) What happened during the response? (number 2) What can we do to improve for next time? (number 4) What was in place before the response? (this is number 1)

Thank you for your time and commitment to the AAR process