Improving Preceptor Narrative Feedback Comments on Student Assessments

Improving Preceptor Narrative Feedback Comments on Student Assessments Suzanne Minor MD; Sarah E Stumbar MD MPH; Marquita Samuels MBA Florida International University Herbert Wertheim College of Medicine Background Methods Learners benefit more from narrative, specific feedback than from feedback consisting solely of numerical ratings. This specific feedback is more easily assimilated, and can lead to improved intraining evaluations. In October 2017, this prompt was added to our Family Medicine clerkship clinical assessment: Narrative comments provided by preceptors are also important in justifying high or low scores and grades, for documenting worrisome behavior, and for use in the Medical Student Performance Evaluation. The quality and quantity of narrative comments can be improved through faculty development about their importance. However, these interventions —including in person visits with faculty, emails, and phone conversations-- are often limited by time constraints. The purpose of our pilot study is to determine if an intervention requiring little devoted time – the addition of a simple prompt to the student clinical assessment – can improve the quality of narrative feedback comments given to our students on the Family Medicine clerkship. “Please comment on one patient encounter the student had that stands out to you (please try to be as descriptive as possible). ” The prompt was added 11/17/2017, and only to our paper assessment, limiting our data to preceptors who use the paper form. We compared the specificity and quality of responses to the new prompt from 11/2017 - to the prior comments received 4/2017 -11/2017, when the existing prompts on the assessment were: “Comment on the strengths of this student’s performance” and “Comment on areas that need improvement for this student. ” After individually rating the comments on specificity and genericity, the three authors met to come to agreement in ratings through an interactive process. Conclusions Results A total of 7 faculty regularly complete paper assessments and were included in this pilot study. Preceptors 1, 2, 3 increased the percent of very specific narrative comments. Preceptor 1 increased to 60% non- generic comments. Preceptors 2, 3, 6 increased the percent of partially (from all) generic comments. This pilot demonstrated that some preceptors, although not all, improved the specificity and originality of their narrative comments with the addition of the new prompt. Our next steps include petitioning the curriculum committee to add this prompt to the electronic assessment for clerkship students. We also plan to target faculty development towards improving the specificity of narrative comments. Specificity of Each Preceptor’s Comments, Pre- and Post-Intervention pre % (n) post % (n) Preceptor NS SS VS 1 90% (9) 10% (1) 0% (0) 20% (1) 60% (3) 2 67% (4) 33% (2) 0% (0) 31% (5) 50% (8) 19% (3) 3 0% (0) 100% (4) 0% (0) 14% (1) 71% (5) 14% (1) 4 50% (3) 5 0% (0) 50% (3) 33% (2) 6 14% (1) 86% (6) 0% (0) 64% (9) 36% (5) 0% (0) 56% (5) 44% (4) 0% (0) Genericity of Each Preceptor’s Comments, Pre- and Post-Intervention Preceptor AG pre % (n) PG 1 2 3 4 5 6 90% (9) 83% (5) 0% (0) 100% (6) 0% (0) 86% (6) 10% (1) 17% (1) 100% (4) 0% (0) 50% (3) 14% (1) NG 0% (0) 50% (3) 0% (0) post % (n) AG PG 40% (2) 38% (6) 14% (1) 75% (9) 33% (2) 67% (6) 0% (0) 62% (10) 86% (6) 25% (3) 67% (4) 33% (3) NG 60% (3) 0% (0) 0% (0) Bibliography 1. Pelgrim EA, Kramer AW, Mokkink HG, Van der Vleuten CP. Quality of written narrative feedback and reflection in a modified mini-clinical evaluation exercise: an observational study. BMC Med Educ. 2012 Oct 18; 12: 97. 2. van der Leeuw RM, Overeem K, Arah OA, Heineman MJ, Lombarts KM. Frequency and determinants of residents' narrative feedback on the teaching performance of faculty: narratives in numbers. Acad Med. 2013 Sep; 88(9): 1324 -31. www. postersession. com