Evaluating Learners Jennifer L Middleton MD MPH FAAFP

Evaluating Learners Jennifer L. Middleton, MD, MPH, FAAFP Academic Medicine Rotation August 2014

Objectives 1. Describe possible goals of evaluating medical learners. 2. List 3 challenges to accurate medical learner challenges evaluation. 3. Define “anchor point. ” anchor point

What is evaluation? Why do we do it? Accreditation (ACGME, AAMC) Assess competence Gauge academic progress Predict performance Guide improvement Assign grades Judge program effectiveness http: //familymed. uthscsa. edu/ACE/chapter 6. htm

What’s the difference between feedback and evaluation?

Provides information about performance Includes judgment about quality Feedback Evaluation YES NO YES Your non-verbal skills showed your patient that you Example were interested in her story. Your doctor-patient communication skills are appropriate for your level of training.

Summative evaluation vs. Formative evaluation during at the end

provides assessment of quality How is formative evaluation different than feedback?

Process performance vs. what an individual does Outcome performance the result Schmutz et al. Five Steps to Develop Checklists for Evaluating Clinical Performance: An Integrative Approach. Acad Med. 2014; 89: 996 -1005

Process performance: Subjective vs. objective measures based on clinical expertise of the rater based on predefined scoring categories using specific criteria Schmutz et al. Five Steps to Develop Checklists for Evaluating Clinical Performance: An Integrative Approach. Acad Med. 2014; 89: 996 -1005

Objective criteria = checklist

Anchor points

Evaluation challenges Grade inflation Halo effect Vague descriptions

One possible process for completing an evaluation: ü ü ü Complete yes/no and/or Likert scale questions first. Expand on any above average or below average scores. Offer specifics re: taking above average skills to the next level. Offer specifics re: improving below average marks. Provide your best estimate of the learner’s capability to improve below average areas.

Creating evaluation forms Summative or formative? Process or outcome? Assess objectives Legible – plenty of room!

Types of evaluation questions Yes/No Likert scale Qualitative What questions would work best to… § Determine if your objectives were met? § Rate skill at a procedure/behavior? § Elicit ideas you might not have thought of?

The 6(7) ACGME (AOA) Competencies Medical knowledge Patient Care Practice Based Learning and Improvement Systems Based Practice Interpersonal Skills and Communication Professionalism Osteopathic Principles and Practices