The Art and Science of Giving Effective Feedback

The Art and Science of Giving Effective Feedback Nur-Ain Nadir. MD. MEHP. FACEP. Kaiser Permanente Central Valley Residency Program Director – Emergency Medicine

Disclosures • None

Objectives At the end of this session, the learner will be able to: 1. 2. 3. 4. 5. 6. Define feedback Discuss the significance of feedback in medical education Distinguish “formative” feedback from summative “evaluation. ” Describe the characteristics of good feedback Describe at least one feedback technique Identify barriers to giving feedback

How do you know you’re doing a good job? “When you don’t call me into your office, I must be doing a good job. ” “I must be doing okay because there haven’t been any speeches about patient safety lately. ” “I must be doing okay because you always tell me when I make too many errors. ”

Definition: Feedback “Feedback is when a learner is offered insight into what he or she actually did, as well as the consequences of his or her actions. Feedback highlights the dissonance between the intended result and the actual result, thereby providing impetus for change. ” Ende, 1983

Definition: Feedback • Specific information provided by another source with the intent to improve performance or understanding • The art of observation (of actions) and description (of outcomes)

Purpose of Feedback To promote learner self-regulation through helping them recognize any discrepancies between what they are doing and what they OUGHT to be doing Rudolph et al. (2007, 2008)

Why Use Feedback? Learners who get effective feedback: do significantly better develop better judgment learn faster Learners rate feedback as one of the most important qualities of a good teacher Most learners want more feedback on their clinical skills Feedback helps everyone Ramani and Krackov, 2012

Without Feedback Mistakes go uncorrected Good performance is not reinforced Clinical competence is not achieved Learners self-validate Ramani and Krackov, 2012

Types of Feedback • Informal (as close to action as possible) – Spontaneous/Unplanned/Brief Example: Student patient presentation • Formal (at the end of the week/month) – Scheduled/Planned/More time Example Mid-Clerkship feedback

Types of Feedback: Formative vs Summative • “Coaching” • Low stakes • Goal = performance improvement • Ongoing • Allows learner the opportunity to correct • Does not impact grade • “Judging” • High stakes • Goal: verification of competency attainment • At the end of rotation • No opportunity to correct • Could potentially impact grade

Feedback • • Presents information Scope: Specific action Goal: Improvement Mostly Formative – i. e. , guides learner to goal • Neutral – i. e. , verbs & nouns vs Evaluation • • Presents a judgment Scope: Global Goal: Grading Typically summative – i. e. , how well learner met goal • Normative statements – i. e. , adverbs & adjectives

National Data Regarding Feedback Only 42% of medical students reported receiving sufficient feedback during core clerkships. AAMC Graduation Questionnaire 2010 -2013

Why are Educators Failing at Feedback? 1) Not learner-centered/without awareness of the learner’s perspective or self-assessment 2) Overpowering of affective reactions to feedback / a failure to separate the behavior and the person (for teacher and/or learner) 3) Unsuccessful feedback teaches learner to fear or avoid feedback in the future Bing-You and Trowbridge, 2009.

Feedback Triad Giving Feedback “Justification” Soliciting Feedback “Lip service” “Demotivation” Receiving Feedback

In the Beginning there needs to be. . . Teaching and Preparation of the Preceptor. . . Feed back is what follows. . .

Before Giving Feedback Have you communicated your expectations? Have you asked how the learner wants to get feedback? Have you conducted an orientation? Have you probed learner’s knowledge, skills, and abilities? Have you checked learner’s experience?

Is Your Environment Conducive to Effective Feedback? Mutual accountability Willingness to learn No fear No surprises Truthfulness Self-responsible language Coaching

Hypothetical Case. Patient admitted to Med-Surg – is 2 days post surgery for lower extremity fracture. Develops shortness of breath. RN requests intern to evaluate.

Hypothetical Case

Probable Responses /Actions Going Through Your Head 1. “Wh. AAAT were you thinking? ” 2. “Misdiagnosis of this is inexcusable. ” 3. “ This is unacceptable. ” 4. Step by step analysis of mistake on work rounds with group. 5. Urge to criticize in group setting in front of peers/patient/staff.

Constructive ways of phrasing/handling it: 1. “Mr X. needs a definite airway right away. Lets intubate right away. ? ” 2. Immediate response: briefly point out error. “We should consider his Airway, Breathing and Circulation and stabilize him before we do anything else. If you noticed he is not responding to usual oxygenation techniques, escalate to an advanced airway. ” 3. Emphasize fixing the problem: “What work-up do we need now? ” -- specific constructive feedback 4. Later go into more detail of errors privately with intern (too humiliating on work rounds).

Characteristics of Good Feedback • • Descriptive, not judgmental Specific, not general Focuses on changeable behaviors Emphasize consequences Value to Learner Timely ( as needed and frequent) Based on first-hand information (not hearsay) Joint action plan with timelines Ende, 1983

Judgmental (not specific) • “That was the resuscitation I have ever seen. I’m tired of watching you screw up. I really don’t think you will do well in this specialty. ”

Specific (not judgmental) Specific things to improve • “I noticed that the patient was in respiratory distress and not responding to oxygen by nasal cannula, despite your turning it up all the way. At times like these, I like to focus on stabilizing the Airway by intubating. I then address the Breathing, and finally the circulation. Once the patient has stabilized I have time to review the differentials. What are your thoughts? ”

Changeable Behavior • Provide feedback on performance discrepancies that are within the control of the learner – Consider current skills / knowledge for learner’s level

Emphasize Consequences • Provide (or ask learner to provide) possible outcomes if the behavior is not changed • Highlight the benefits to changing the behavior

Be Timely • Ensures more accurate recall • Immediate feedback is more believable • Allows learner to modify behavior earlier versus letting it continue (making it more difficult to correct)

Bite Sized Feedback • Make your feedback digestible • You may observe plenty wrong with a learner’s performance • Choose the top 2 -3 most significant aspects of performance to address • Any more and it will be lost in translation. • Annotate it to bring it back up at a later time such as a formal mid clerkship feedback session.

Based on Firsthand Information • Observe the student • You cannot provide feedback on – Skills you haven’t observed – Knowledge you haven’t questioned – Reasoning you haven’t heard – Attitudes you haven’t experienced

Example 1. Student Too Slow • Instead of – “You took way too long interviewing that patient. You need to be out of the room in about 20 minutes with your H&P. ” • Say • “I noticed you were in Mrs. Smith’s room for an hour. ” • Then wait

Example: Student Too Slow • Student replies – “I know, I just couldn’t get the patient to stop talking!” – “I didn’t know which questions were most important to ask” – “I spent 40 minutes doing the physical exam” • Recommend how to improve

Example 2. Student Mistake • Instead of – “You wanted to give the patient the wrong antibiotic for their pneumonia” • Say – “Why did you want to give the patient Keflex for their pneumonia? ” • Then wait (silence = open ended invitation for student to self-reflect/critique and formulate a response)

Example: Student Mistake • Student replies – “You mean Keflex isn’t a good antibiotic for pneumonia? ” – “oops, I meant to click on ceftriaxone instead. ” – “Dr. Attending said to put them on Keflex. I thought it was a bad idea. ”

Example: Student Mistake • Recommend how to improve – Read the IDSA’s recommendations for CAP. You can find them… – Computer order entry can be dangerous if you don’t pay close attention to what you are ordering… – I’ll clarify this with Dr. Attending

Example 3: Disorganized Student Presentation • Instead of: “You need to improve your presentation. ” (not specific) • Say: “This morning on rounds, your presentation was very disorganized. It was difficult to follow your thought process. ” • Wait – student can tell you what the problem was from their standpoint

Example: Disorganized Student Presentation • Recommend how to improve – “When you have a complicated patient, organize your presentation with a problem list. That way others can follow your thought process and your presentations will be more concise and organized. ” • “Also, on rounds, listen to how the stellar interns present and try to model your presentations like theirs. ”

Feedback Set Up, Techniques, Scripts

Anatomy of a Feedback Session Bienstock et al. , 2007

“The Feedback” • • Pendleton’s Sandwich Ask – Tell – Ask Sandwich SOAP technique Advocacy-Inquiry

Pendleton’s rule a. k. a Kiss-Kick-Kiss Sandwich 1. Praise 2. Critique 3. Praise Pendleton et al. , 1983

Ask-Tell-Ask Sandwich • ASK: Ask learner to assess own performance first • • Begins a conversation – an interactive process Assesses learner’s level of insight and stage of learning Less threatening: separating behavior from “self” Promotes reflective practice • TELL: Tell what you observed: diagnosis and explanation • • • React to the learner’s observation • Feedback on self-assessment Include both positive and corrective elements • “I observed…. ” Give reasons in the context of well-defined shared goals • ASK: Ask about recipients understanding and strategies for improvement • • • What could you do differently? (Elicit learners suggestions for improvement) Give your suggestions for improvement Paranjape, 2002 Commit to monitoring improvement together (plan)

SOAP technique: Diagnose your Learner Subjective: (Learner’s) Subjective self-assessment (Ex. How did it go? What did you think you did well? What could you have done differently? ) Objective: Your Objective Feedback on Specific Actions/Events (Ex. I noticed that you appeared uncomfortable when obtaining the sexual history. Can you tell me what was happening? ”) Assessment: Assess and Summarize – Learning. Ask learner to give you two take home points. Plan: Formulate an Action Plan for improvement

Oh Crap !! Scenarios. . • Adverse Events/Near Adverse Events/Medical Errors/Patient Safety issues • Piss you off!!! (Emotional reaction from you the instructor) • Excellent teaching moments • Excellent feedback moments. • Excessively DIFFICULT moments.

Learning Principles For cognitive brain to work: Learner: Psychological safety* ‘behave or perform without fear of negative consequences to self-image, social standing, or career trajectory. ’’ PFC Thinking brain Needs Motivations Stressors Respect Instructor: Self-awareness Control Limbic System Primitive, emotinoal Flight fright, Freeze https: //www. sketchport. com/drawing/576148803 9968768/brain

Formal or Corrective Feedback • Think it through first – Control your emotions – Anticipate reaction • Private settings • Explain the reason for the feedback – Focus on actions/behaviors – Supported by objective examples and direct observation – Stay calm, use conversation voice

Formal or Corrective Feedback • Actively listen to responses • Clarify any misunderstandings • Plan to move forward

Borrowing from Simulation Debriefing Pedagogies Real-Time Debriefing • Debriefing as Formative Assessment and Source of Feedback

Sim Debrief Learner elicited Explorative Thought process behind actions Safe space Real-Time Debrief/ Feedback Learning: Experiential Reflective KSA Gap Instructor elicited Directive Action analysis Real world

Advocacy-Inquiry Feedback Script “ I noticed/observed (action/actions) ___” “ According to XYZ criteria (e. g ACLS guidelines) it should have been managed this way. ” “I am curious. What are your thoughts? ”/ “What happened? ”/ “What’s up? ”

Barriers to Giving Feedback “The learners already know how they’re doing. ” “I don’t have time. ” “That must have been an anomaly. ” “I’m not used to this. I feel awkward. ” “Learners get defensive. ” “The learner isn’t ready to receive feedback. ” “Who/what do I compare them to? ”

Summary Effective feedback is essential to both learning and good teaching Learners who receive useful feedback perform better Without feedback, mistakes go uncorrected Feedback is a skill and can be learned & improved upon

Questions? ? ?

References • • • Ende J. Feedback in Clinical Medical Education. JAMA 1983; 250: 777 -781. Bing-You RG, Trowbridge RL. Why medical educators may be failing at feedback. JAMA 2009; 302: 1330 -1331. Rudolph JW, Simon R, Raemer DB, et al. Debriefing as formative assessment: Closing performance gaps in medical education. Acad Emerg Med. 2008; 15(11): 1010 -1016. Bienstock, JL, Katz, N, Hueppchem, N, Erickson, S and Puschek, E. 2007. To the Point, Medical Education Reviews: Feedback. American Journal of Obstetrics and Gynecology. 2007; 508 -513. Branch J, Paranjape A. Feedback and Reflection: Teaching Methods for Clinical Settings. Academic Medicine. 2002; 77: 1185 -1188. Ramani S and Krackov S. Twelve tips for giving feedback effectively in the clinical environment Medical Teacher 34: 787 -791