Competencybased Medical Education An Overview with Attention to

Outline Definition of CBME • Frameworks • Outcomes Implication for GME training Assessment and evaluation strategies • Narratives • Entrustable Professional Activities (EPAs) • Milestones

® What does competency-based medical education means to you? © 2008, 2009 American Board

Competency-Based Medical Education is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies The International CMBE Collaborators 2009

Framework - ACGME General Competencies Medical knowledge Patient care and procedural skills Professionalism Interpersonal and communication skills Practice-based learning and improvement Systems-based practice

So What is the Outcome and Who Determines it? The Profession? The Public? Policy Makers?

The Profession? • The “core” of Internal Medicine? • Competence in the six ACGME general competencies? • Safe and effective patient care? The Public? • Trust that a doctor can do certain things? Policy Makers? • Meeting the needs of the complex and aging US health population? • Medicare goals?

Traditional versus CBME: Start with System Needs Frenk J. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010 8

A new paradigm In Competency-Based Medical Education (CBME) we must truly know the trainee has demonstrated competence and is ready to progress to the next stage of their career: • Requires clear definition of expected competencies • Requires valid and reliable assessment and evaluation to confirm competency is consistently demonstrated in the clinical environment

The CBME Basics Outcome Data Assessment Curriculum

“Unsupervised Safe and Effective Patient Care Data Assessment Curriculum

Judgment or Attestation Evaluation – judgment about competence based upon available assessment data. Who makes the judgment that the outcome has been reached? • The literature supports the benefit of: • Group (competency committees) rather than individual decisions. Schwind, (2004), Williams, (2005) Thomas (2011) • Narratives describing learners rather than numbers. Regehr, (2007), Crossley (2011)

“Unsupervised Safe and Effective Patient Care Portfolio of Criteria Referenced Data Assessment Curriculum

Assessment and Evaluation System: Components Clinical Competency Committee ·Periodic review – professional growth opportunities for all ·Early warning systems Advisor Structured Portfolio ·Entrustment-based assessment OSCE Mini-CEX or CSR Medical record audit/QI project Clinical question log Multisource feedback ·Monthly Evaluations ·Research Project ·Trainee contributions (personal portfolio) Trainee ·Review portfolio ·Reflect on contents ·Contribute to portfolio Summative Assessment Process - Fas. Track Licensing, Certification, and Accreditation · ABIM · ACGME Program Leaders ·Review portfolio periodically and systematically ·Develop early warning system ·Encourage reflection and self-assessment

“Safe and Effective Patient Care Portfolio of Criteria Referenced Data Entrustment Focused Assessment Curriculum

Assessment Challenges Ensure that assessment and evaluation of milestones document competence in those activities that define the profession – (the outcome!) • Entrustable professional activities or EPAs? • What the public (and the profession) trust physicians are capable of doing.

Entrustable Professional Activities EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialty • Why is a fellowship trained geriatrician different that a general internist? The concept of “entrustable” means: • ‘‘a practitioner has demonstrated the necessary knowledge, skills and attitudes to be trusted to independently perform this activity. ’’ 1 1 Ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007; 82(6): 542– 547.

“Entrustment in GME” What do faculty and training programs “entrust” trainees to do? • Progressive independence while delivering patient care • Family meetings – determining “goals of care” Capture multiple competencies “work-based” entrustments serve as foci for assessment and evaluation What justifies these “entrustments”?

EPA in Practice Individual faculty member making entrustment decision for a specific trainee • Level 1 – not allowed to practice EPA • Level 2 – practice with full supervision • Level 3 – practice with supervision on demand • Level 4 – “unsupervised” practice allowed • Level 5 – supervision task may be given ten Cate et al, 2007

EPA in Practice Program director with competency committee • Determine resident progression to next steps of training • Attestation to accreditation or certification bodies regarding developmental progression • Attestation to public that resident is entrusted to practice independently

“Safe and Effective Patient Care” Portfolio of Criteria Referenced Data Entrustment Focused Assessment Milestones

Milestones A significant point in development. The IM milestones • are organized by the ACGME general competency domains • define the abilities (K/S/A) expected of IM residents as they progress through training • Framed in behavioral terms • They are observable • Sets the stage for assessment of competence

Patient Care ACGME Competency Developmental Milestones Informing ACGME Competencies Clinical skills and reasoning Manages patients using clinical skills of interviewing and physical examination Historical Data Gathering 1. Acquire accurate and relevant history from the patient in an efficiently customized, prioritized, and hypothesis driven fashion 2. Seek and obtain appropriate, verified, and prioritized data from secondary sources (e. g. family, records, pharmacy) 3. Obtain relevant historical subtleties that inform and prioritize both differential diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient RRC sub-bullet Approximate Time Frame Trainee to Achieve Stage 6 months 9 months 18 months Assessment Methods/Tools Standardized patient Direct Observation Simulation

Tracking the Learner Fully Competent A’ A B C Start PGY 1 EPA based assessment Finish PGY 3 Lucey and Boote

® Putting It All Together Entrustable Professional Activities, Entrustment in GME, and Assessments ©

Attestation of Competency In Desired Outcome Portfolio (Entrustment/ EPA generated assessment data) Entrustment -focused Assessment Selected key milestones Entrustment-focused Assessment Selected key milestones Milestones = discrete K/S/A expected of learners as they progress through training

Lead a Health Care Team Lead a Resident Team (Competency Committee) Multisource Feedback Direct Observation Chart Stimulated Recall (Health Care Team) (Core Faculty) IPCS-Effective team communication SBP-Understands roles of team and system PBLI-Welcomes feedback P-Communicates feedback to health care team IPCS-Role model/teach effective communication during transitions of care PBLI-Actively participate in teaching conferences SBP-Minimize unnecessary care PBLI-Classify and articulate clinical questions PC-Develop prioritized DDx for common inpt and outpt conditions

Code Team Leader (18 month) COMPETENCE Distance Supervision in Ambulatory (6 month) Ward Supervisor (24 month) Nightfloat (12 month) Running Family Meeting (30 Month) Caverzagie

The Pace of Change?