Defining and Assessing Entrustable Professional Activities Karen E

Defining and Assessing Entrustable Professional Activities Karen E. Hauer, MD H. Carrie Chen, MD, MSEd John Q. Young, MD, MPP Patricia S. O’Sullivan, Ed. D With thanks to Olle ten Cate, Ph. D UCSF June 5, 2013

Agenda 1 -1: 15 1: 45 2: 05 2: 35 Break 3: 15 3: 45 4: 00 4: 30 Welcome Introduction to EPAs individual writing of an EPA overview of EPA development small group activity, design an EPA (steps 1 -4) small group reporting overview of steps 5 -7 second activity, evidence for an EPA (steps 5 -7) small group reporting and wrap up (all)

Workshop objectives By the end of this workshop, participants will be able to: Define an entrustable professional activity (EPA) Develop an entrustable professional activity that can be used for assessment of medical trainees

What do you hope to learn today?

Current Assessment System ACGME core competencies: Patient Care Medical Knowledge Practice-based Learning and Improvement Interpersonal and Communication Skills Professionalism Systems-based Practice

Current Assessment System Practice-based Learning and Improvement (Housestaff) (Question 5 of 9 - Mandatory) Assimilates evidence from scientific studies or consultants related to their patients' health problems. Participates in organized transitions of care curriculum as specified at each site. Actively seeks out and acts on performance feedback from the supervising attending and other team members. Insufficient Contact to Judge 0 1 & 2 = Unsatisfactory 3 = Marginal 1 2 3 Satisfactory/Meets Expectations 4 5 6 7 & 8 = Excellent 9 = Outstanding 7 8 >> 9 <<

Current Assessment System What are some limitations of current assessment systems? Core competencies and sub-competencies: long checklists of behavioral descriptors Relies on traditional but limited assessment methods – knowledge exams, ward evaluations Not a holistic summative view of the trainee

Competency An observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes. The International CBME Collaborators, 2009

Competent Possesses the required abilities in all domains in a certain context at a defined stage of medical education or practice. The International CBME Collaborators, 2009

Competence entails more than the possession of knowledge, skills and attitudes; it requires you … to apply these [abilities] in the clinical environment to achieve optimal results. ten Cate, Med Teach, 2010 does the job

Can you trust the learner to function independently?

How to Drive a Car Competency Can accelerate and brake smoothly Can approach an intersection and can turn left Competent Passes driver’s education classes Passes driver’s exam to get the license Competence Drives safely on interstate or during bad weather, avoids accidents, no traffic tickets Parents hand over the keys and walk away

What is the goal with assessment? Integrated, longitudinal, learner-centered assessment system Promote skill acquisition in multiple domains concurrently Assess what learners actually do in practice Be able to conclude: this is a trustworthy trainee

Features of genuine competency based medical education Outcome-based, not process-based: what is attained is key, not just what is done or taught Integrates knowledge, skill, attitude Time-independent: length of training for defined outcomes is not pre-set Individualized: trainees and contexts vary Workplace-learning based Lifelong learning oriented

Competence. What criteria would you use to select your doctor? Passed all tests and exams? Grades and scores? Years of training? Follows protocols and guidelines? Trust that s/he will manage a case in the best possible way?

Entrustable Professional Activities (EPAs)

Entrustable Professional Activity (EPA) Define important clinical activities Link to competencies / milestones Include professional judgment of competence by clinicians Make “decisions of entrustment” for “entrustable” activities Something changes for the learner…. Build collection of mastered EPAs (portfolio) to document full competence

Definition of an Entrustable Professional Activity A core unit of work, reflecting a responsibility that should only be entrusted upon someone with adequate competencies More specific: Part of essential professional work in a given context Independently executable, within a time frame Leads to recognized output of professional labor Observable and measurable in process and outcome, leading to a conclusion (“well done” or “not well done”) Must require sufficient, specific knowledge, skill and attitude, generally acquired through training Should reflect competencies, important to be acquired Usually confined to qualified personnel only

Competencies versus EPAs • • • Competencies person-descriptors EPAs work-descriptors knowledge, skills, attitudes, values essential parts of professional practice content expertise collaboration ability communication ability management ability professional attitude scholarly approach • • • discharge patient counsel patient lead family meeting design treatment plan perform paracentesis resuscitate if needed

Competencies and EPAs combined xx x xx Lead a family meeting to discuss serious news with patient/family and other health providers xxx x x xx x Practice x System-Based Professionalism xxx PBLI Communication Medical Knowledge Patient Care Develop and implement a safe discharge plan for a patient from the acute care setting

Exercise Complete Worksheet A on your own List good examples of essential EPAs your trainees should be able to do independently at the end of training. List bad examples of EPAs Pair Share: share your examples with one person next to you.

Recommended full EPA description 1. EPA Title (max 20 words, avoid skill and avoid adjectives) 2. Description of the activity (to serve universal clarity, include limitations) 3. Expected KSA (to serve trainee) 4. Link with competencies and predefined milestones (to embed within the existing framework) 5. Sources of information to determine progress (to serve observation and assessment) 6. Basis formal entrustment decision (who will have a say in the decision -- signatures if formal and documented) 7. Post level-4 of entrustment (“unsupervised”) (what difference does it make for the trainee? )

Example from Pediatrics One of the 16 draft EPAs developed by AAMC and ABP team for pediatric residency training: 1. EPA Title: Manage patients with acute, common single system diagnoses in an ambulatory, emergency, or inpatient setting 2. Description: Merged with expected KSA and clarified in descriptive vignettes

Example EPA cont’d 3. Expected KSA: Gathering info thru hx, PE, and initial labs Sound clinical reasoning driving development of DDx to allow proper diagnostic testing and initial therapy Knowledge of evidence related to primary problem Application of evidence to management plan Patient and family-centered care with bidirectional communications Documentation of plan and reasoning that is transparent to other members of health care team

Example EPA cont’d 4. Link with competencies/milestones: Patient care Perform complete and accurate PE Make informed diagnostic and therapeutic decisions that result in optimal clinical judgment Develop and carry out management plan Medical knowledge Interpersonal and communication skills Communicate effectively with patients, families, and the public as appropriate, across a broad range of socioeconomic and cultural backgrounds Maintain comprehensive, timely, and legible medical records

PE Sub-competency Performs essentially the same rote head-to-toe physical examination of the patient regardless of presenting complaint; does not use diagnostic hypotheses from the history to anticipate or look for specific positive or negative findings on physical examination. With a broad list of diagnostic hypotheses after the history, uses a head-totoe approach to the physical examination to anticipate and look for a myriad of potential positive and negative physical examination findings for multiple diagnostic considerations. This approach can lead to failure to identify pertinent and important physical findings that are present, misinterpretation of physical findings, and attribution of importance and meaning to irrelevant findings. Uses a narrow list of diagnostic hypotheses generated through the history to anticipate and look for specific positive or negative physical examination findings of only the most relevant diagnostic considerations; open to new diagnostic possibilities in the process of performing a survey physical examination to elicit unexpected abnormalities but may dismiss these as unimportant when it is difficult to integrate these findings into the working differential diagnosis. Uses a narrow list of diagnostic hypotheses generated through the history as well as through extensive clinical experience to anticipate and look for key specific physical examination findings that will discriminate between competing similar diagnoses; uses surprises that result from a survey physical examination to rethink and retest diagnostic hypotheses; actively looks for physical exam findings that disconfirm the working diagnosis or rule in or out rare but high-risk alternative diagnoses.

Description Behaviors of early, more advanced, competent, proficient, and expert learner Expected elements from milestones Vignette – 2 yo with wheezing, resp distress Reassurance provided during hx Has FB aspiration on differential so focuses on differential BS in addition to wheezing and WOB Presents focused hx/PE with reasoned assess/plan consistent with family’s wishes and health literacy, including SW referral for loss of insurance

Small Group Activity: Design an EPA Work in small groups Use resources provided Internal medicine milestones SOM milestones Develop an EPA for Internal medicine OR Medical students

Small Group Debrief

When is “competence” reached? When you trust the trainee When a professional activity is mastered • on a threshold level • that permits unsupervised practice • and full entrustment It happens all the time: when trainees work without direct supervision

Level of supervision Level 1: not allowed to practice the EPA Level 2: practice with full supervision Level 3: practice with supervision on demand Level 4: “unsupervised” practice allowed Level 5: supervision task may be given

Recommended full EPA description 1. EPA Title (max 20 words, avoid skill and avoid adjectives) 2. Description of the activity (to serve universal clarity, include limitations) 3. Expected KSA (to serve trainee) 4. Link with competencies and predefined milestones (to embed within the existing framework) 5. Sources of information to determine progress (to serve observation and assessment) 6. Basis formal entrustment decision (who will have a say in the decision -- signatures if formal and documented) 7. Post level-4 of entrustment (“unsupervised”) (what difference does it make for the trainee? )

Dreyfus and Dreyfus Model • Novice – Don’t know what they don’t know • Advanced Beginner – Know what they don’t know • Competent – Able to perform the tasks and roles of the discipline – restricted breath and depth • Proficient – In depth knowledge concerning the discipline – often rule based – know what they know • Expert – Expert thrives with situations that break the rules – who the proficient practitioners go to for help

Development of Competence expert proficient competent advanced novice training deliberate professional practice Dreyfus & Dreyfus, 1986

Sample Competency Curve EPA 2 EPA 4 EPA 3 EPA 1 Competence EPA 5 Threshold Justified entrustment decisions training deliberate professional practice

Small Group Activity: Choosing Evidence What evidence will inform the EPA that you developed

Small Group Reporting

Wrap Up