Accreditation Council for Graduate Medical Education The Next

Accreditation Council for Graduate Medical Education The Next Accreditation System, and The Clinical Learning Environment Review (CLER) Where We Are In Implementation Thomas J. Nasca, MD MACP Chief Executive Officer © 2013 Accreditation Council for Graduate Medical Education (ACGME)

Disclosure • TJN: Professor of Medicine, Jefferson Medical College • Full Time Salaried by ACGME • No conflicts of interest to report

What Currently Drives the Structure and Content of our Residency Programs? In the context of local service needs, Choose Educational Experiences within Institution, Faculty “Educate” “Curriculum” Residents ACGME Standards Identify/Develop Evaluation Idiosyncratic Tools - Formative and Summative - Experience Tracking Guarantees that education is institutionally idiosyncratic, and lags rather than anticipates change in the delivery system “Circumstantial Practice” © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Is this model producing physicians ready to practice in the future (2025 and beyond)? Are there areas for us to improve?

Landrigan, C. P. , et. al. NEJM 2010; 363: 2124 -34

The North Carolina Experience Temporal Trends in Rates of Patient Harm Resulting from Medical Care, 2002 -2007 Landrigan, C. P. , et. al. NEJM 2010; 363: 2124 -34 • No significant change in: • All Harms • Preventable Harms • High-Severity Harms • …whether evaluated by external or internal reviewers

From: Evaluating Obstetrical Residency Programs Using Patient Outcomes Asch, DA, et. al. , JAMA. 2009; 302(12): 1277 -1283. doi: 10. 1001/jama. 2009. 1356 Copyright © 2012 American Medical Association. All rights reserved.

Evaluating Residency Programs Using Patient Outcomes n= 4, 906, 169 deliveries in Florida and New York, 1992 -2007 4124 physician program graduates of 107 residency programs Difference remains after correction for USMLE performance Excess Risk ∆ 33% Q 1 vs Q 5 ∆ JAMA 2009; 302(12): 1277 -1283. Asch, DA, et. al. , Table 4

“All systems are perfectly designed to get the results they are getting. ” Various Attributions: Paul Batalden MD Donald Berwick MD W. Edwards Deming

Despite the many successes of the American GME Effort. . . Suffice it to say that there is sufficient motivation for change…

There are no easy solutions or shortcuts! “I have a microwave fireplace. You can lay down in front of the fire all night in eight minutes. ” Steven Wright

Intellectual Foundation of the Educational Innovations Project (EIP) A New Model for Accreditation of Residency Programs in Internal Medicine Allan H. Goroll, MD; Carl Sirio, MD; F. Daniel Duffy, MD; Richard F. Le. Blond, MD; Patrick Alguire, MD; Thomas A. Blackwell, MD; William E. Rodak, Ph. D; and Thomas Nasca, MD, for the Residency Review Committee for Internal Medicine Ann Intern Med. 2004; 140: 902 -909. © 2012 Accreditation Council for Graduate Medical Education (ACGME)

“Task Force on Patient Safety and Professionalism” Nasca, T. J. , Day, S. H. , Amis, E. S. , for the ACGME Duty Hour Task Force. Sounding Board: The New Recommendations on Duty Hours from the ACGME Task Force. New England Journal of Medicine. 362 (25): e 3(1 -6). 2010. June 23, 2010.

The actions of the ACGME must fulfill the social contract, and must cause sponsors to maintain an educational environment that assures: • the safety and quality of care of the patients under the care of residents today • the safety and quality of care of the patients under the care of our graduates in their future practice • the provision of a humanistic educational environment where residents are taught to manifest professionalism and effacement of self interest to meet the needs of their patients © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Nasca, T. J. , Philibert, I. , Brigham, T. P. , Flynn, T. C. The Next GME Accreditation System: Rationale and Benefits. New England Journal of Medicine. Published Electronically, February 22, 2012. In Print, March 15, 2012. DOI: 10. 1056/nejmsr 1200117 www. nejm. org. NEJM. 2012. 366; 11: 1051 -1056. 1

The Building Blocks or Components of The “Next” Accreditation System 10 year Self-Study Visit 10 year Self-Study prn Site Visits (Program or Institution) Continuous RRC and IRC Oversight and Accreditation Clinical Learning Environment Review CLER Visits © 2012 Accreditation Council for Graduate Medical Education (ACGME)

2010 -2011 National Advisory Committee Advise ACGME on establishment and goals of the Sponsor Site Visit Program Carolyn Clancy, MD James Bagian, MD PE Kevin Weiss, MD MPH Joann Conroy, MD John Combes, MD Paul Schyve, MD Donald Goldmann, MD David Nash, MD MBA Carl Patow, MD MPH Robert Wachter, MD Timothy Flynn, MD Baretta Casey, MD MPH Director, Agency for Healthcare Research and Quality Director, Veterans Administration National Patient Safety Center President, American Board of Medical Specialties Chief Health Care Officer, Association of American Medical Colleges Senior Vice President, American Hospital Association Senior Vice President, The Joint Commission Senior Vice President, Institute for Health Care Improvement Dean, School of Population Health, Thomas Jefferson University Vice President, Health Partners Institute for Medical Education Associate Chairman, Department of Medicine, Univ. of California, San Francisco Senior Associate Dean for Clinical Affairs, Chief Medical Officer, Univ. of Florida and Shands Medical Center Director for the Univ. of Kentucky Center for Excellence in Rural Health John Duval, MBA Chief Executive Officer, Medical College of Virginia Hospitals Timothy Goldfarb Chief Executive Officer, Shands Medical Center Paige Amidon, MBA Vice President, Health Programs. Consumers Union Carmen Hooker Odom, MRP President, Milbank Memorial Fund Thomas Nasca MD ACGME Convener

Emphasis of CLER Assessment of Effectiveness of Sponsor in: • integration of residents into Patient Safety programs of the institution, and demonstration of impact • integration of residents into Quality Improvement programs of the institution, efforts to reduce Disparities in Health Care Delivery, and demonstration of impact • establishment and implementation of Supervision policies • oversight of transitions in care • oversight of duty hours standards implementation • Emphasis on Professionalism throughout © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Integration of CLER into The Next Accreditation System • Provides frequent onsite sampling of the learning environment • Permits less frequent onsite scrutiny of high functioning programs • Emphasizes elements of “new” competencies demanded by the public • Accountability • GME Sponsors demonstrate leadership in Patient Safety, Quality Improvement, and Reduction in Disparities • Affirms our commitment to the Social Contract • Move from “duty hours” to Patient Care Quality and Safety © 2012 Accreditation Council for Graduate Medical Education (ACGME)

The Building Blocks or Components of The “Next” Accreditation System 10 year Self-Study Visit 10 year Self-Study prn Site Visits (Program or Institution) Continuous RRC and IRC Oversight and Accreditation Clinical Learning Environment Review CLER Visits © 2012 Accreditation Council for Graduate Medical Education (ACGME)

The “Next Accreditation System” in a Nutshell • Continuous Program Accreditation Model – annually updated • Based on annual data submitted, other data requested, and program trends • 10 year Self Study and Self Study Visit • Standards revised every 10 years • Standards Organized by Core and Detailed: • • Structure Resources Processes Outcomes © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Standards Implementation Across the Continuum of Programs in a Specialty Initial Accreditation New Programs Accreditation with Warning Maintenance of Accreditation New Programs, Accredited Programs with Major Concerns Accredited Programs without Major Concerns Probationary Accreditation Maintenance of Accreditation with Commendation Core and Detailed: Structure Resources Process Outcomes Core: Structure Resources Process Outcomes STANDARDS Core and Detailed: Structure Resources Process Outcomes Withhold Accreditation Withdrawal of Accreditation © 2012 Accreditation Council for Graduate Medical Education (ACGME)

All 9, 022 ACGME Accredited Residency and Fellowship Programs* January 1, 2013 * Excludes programs with Initial Accreditation @ 2013 Accreditation Council for Graduate Medical Education (ACGME)

All 9, 022 ACGME Accredited Residency and Fellowship Programs* January 1, 2013 95. 7% 4% 0. 3%, n=27 * Excludes programs with Initial Accreditation @ 2013 Accreditation Council for Graduate Medical Education (ACGME)

Standards Implementation Across the Continuum of Programs in a Specialty Initial Accreditation New Programs STANDARDS Core and Detailed: Structure Resources Process Outcomes Accreditation with Warning Maintenance of Accreditation New Programs, Accredited Programs with Major Concerns Accredited Programs without Major Concerns Probationary Accreditation Maintenance of Accreditation with Commendation 4. 3% 95. 7% Core and Detailed: Structure Resources Process Outcomes Withhold Accreditation Withdrawal of Accreditation © 2012 Accreditation Council for Graduate Medical Education (ACGME) Core: Structure Resources Process Outcomes

? The Potential Impact of Reduction in Federal Funding in the United States: A Study of the Estimates of Designated Institutional Officials. Nasca, T. J. , Miller, R. S. , Holt, K. D. Journal of Graduate Medical Education. 2011; 3(4): 585 -590. Health Policy Report. The Uncertain Future of Medicare and Graduate Medical Education. Iglehart, J. NEJM (10. 1056/NEJMhpr 1107519) Published on September 7, 2011, at NEJM. org.

Annual Data Collection Focus on Existing Data • Annual ADS Update • • • Resident and Faculty Information Major Changes Citation Response Program Characteristics – Structure and Resources Scholarly Activity – New (from CV to data driven) Block Diagram - New Board Pass Rate Data (external) Resident Clinical Experience Resident Survey Faculty Survey - New Semi-Annual Resident Evaluation • Q 6 month evaluation, Milestone Reporting - New Reporting © 2013 Accreditation Council for Graduate Medical Education (ACGME)

The Goal of the Continuum of Clinical Professional Development Master Expert Proficient Competent Advanced Beginner Novice Undergraduate Medical Education Graduate Medical Education © 2012 Accreditation Council for Graduate Medical Education (ACGME) Clinical Practice

The Goal of the Continuum of Professional Development in the 4 year preparation of the Anesthesiologist Master Expert Anesthesia Related Technical Skills System Based Practice, OR Team Skills Patient Care, Non-Procedural Proficient Competent Advanced Beginner Novice PGY 1 PGY 2 PGY 3 PGY 4 MOC Increase the Accreditation Emphasis on Educational Outcomes © 2012 Accreditation Council for Graduate Medical Education (ACGME)

The Goal of the Continuum of Professional Development in the 4 year preparation of the Anesthesiologist Master Anesthesia Related Technical Skills Expert Proficient Competent Advanced Beginner Novice PGY 1 PGY 2 PGY 3 PGY 4 MOC Increase the Accreditation Emphasis on Educational Outcomes © 2013 Accreditation Council for Graduate Medical Education (ACGME)

Milestones • Observable developmental steps moving from Novice to Expert/Master • “Intuitively” known by experienced medical educators in each specialty • Organized under the rubric of the six domains of clinical competency • Describe a trajectory of progress from neophyte towards independent practice • Articulate shared understanding of expectations • Set aspirational goals of excellence • Provide a framework and language for discussions across the continuum © 2012 Accreditation Council for Graduate Medical Education (ACGME)

2010 -2012 Singapore Mid and End PGY-1, Mid and End PGY-2 Year Evaluation, Mean Overall Rating of Six Competencies across All Specialties Expert Professionalism Mean ± SEM Proficient Communication Skills Medical Knowledge Competent Patient Care Advanced Beginner Practice Based Learning and Improvement Systems Based Practice Novice PGY 1 (0. 5) n= 398 PGY 1 (1. 0) 400 PGY 2 (1. 5) 122 PGY 2 (2. 0) 120 Increase the Accreditation Emphasis on Educational Outcomes © 2013 ACGME International

Singapore End PGY-1, Mid PGY-2, End PGY-2 Milestone Data, by Resident (First Cohort) Professionalism Medical Knowledge Practice Based Learning and Improvement 9 9 9 8 8 8 7 7 7 6 6 6 5 5 5 4 4 4 3 3 3 2 2 2 1 1 1 0 0 T 1 Pr T 2 Pr T 3 Pr 0 0 T 1 CS T 2 CS T 3 CS Communication Skills © 2013 ACGME International 9 T 1 MK T 2 MK T 3 MK 0 T 1 PC T 2 PC T 3 PC Patient Care Technical Skills 0 T 1 PBL T 2 PBL T 3 PBL T 1 SBP T 2 SBP T 3 SBP System Based Practice

ACGME Goal for Milestones - Permits fruition of the promise of “Outcomes Based Accreditation” • Tracks what is important - Outcomes • Begins using existing tools and observations of the faculty • Clinical Competency Committee triangulates progress of each resident • Essential component of a valid and reliable clinical evaluation system • ABMS Board has the opportunity to track the identified individual • ACGME Review Committee tracks unidentified individuals’ trajectories © 2012 Accreditation Council for Graduate Medical Education (ACGME)

ACGME Goals for Milestones “Cohesion for the Continuum” • Able to provide accountability for effectiveness of educational program in producing outcomes • ACGME can work with: • AAMC, LCME to focus graduation level preparation • ABMS, AHA, ACCME, others to identify areas for milestone improvement at graduation from residency/fellowship Milestones Premedical Education BA/BS Medical Education MD Specialty Education (Residency) © 2012 Accreditation Council for Graduate Medical Education (ACGME) Subspecialty Education (Fellowship) Continuing Education/ (MOL – MOC) Graphic After D. Kirch, M. D. , with permission

The Continuum of Development of the Physician in the USA Novice Transition to Practice Transition to Residency ACGME Transition to Medical School AMA Cat 1 Credit System Performance in Practice Transition to College ABMS – Certification, MOC FSMB – Licensure, MOL -V is a )L ic en se ’s LCME C FS MS M S/ B A EC – BM FM (Tra S G ining – ITE Collegiate Accreditation Advanced Beginner Competent Proficient Expert Master ACCME NBME FSMB MCAT SAT Transition to Retirement AHA AHME AIAMC COTH/AAMC

The Building Blocks or Components of The “Next” Accreditation System 10 year Self-Study Visit 10 year Self-Study prn Site Visits (Program or Institution) Continuous RRC and IRC Oversight and Accreditation Clinical Learning Environment Review CLER Visits © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Self Study A Departmentally Coordinated Effort • Respond to any Active Citations • Evaluate Programmatic Performance against Goals • Review Previous 10 year “Annual Program Evaluations” (APE’s) • Demonstrate effectiveness of modifications of the Program over time • Establish Programmatic Goals for the future © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Self Study Visit (Draft) • Team of site visitors • Review the Self Study of the Departmental Educational Effort (Core and Subs) • Conduct a “PIF-less” Site Visit • Validate Annual Data submitted • Potentially serve as a vehicle for: • Description of Salutary Practices • Accumulation of Innovations in the field © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Implementation Timeline for The “Next” Accreditation System • First “round” of CLER Visits – underway • 9/2012 -6/2014 • NAS Policies and Procedures take effect 7/1/2013 • Phase 1 Specialties • 1/2012 Ceased routine Site Visits • 7/2012 Entered “preparation year” • 7/2013 Begin Continuous Accreditation • Phase 2 Specialties, Institutional, Transitional Year • 1/2013 Ceased routine Site Visits • 7/2013 Will enter “preparation year” • 7/2014 Will begin Continuous Accreditation • Self Study Visits begin 2015 © 2012 Accreditation Council for Graduate Medical Education (ACGME)

What Will Drive the Structure and Content of our Residency Programs in the Near Future? Needs of Patients and the Public The Required Outcomes in Each Clinical Competency (Milestones) Design Educational Experiences Select Faculty Expert Physicians who aspire to Mastery (Outcomes) National Evaluation Tools to Track Outcomes - Formative and Summative - Clinical Outcomes Tracking (not just counting) Guarantees that education has the opportunity to anticipate change in the delivery system “Deliberative Practice”

“The Future ain’t what it used to be!” Yogi Berra New York Yankees Catcher, Philosopher

Optimism “What lies behind us and what lies before us are tiny matters compared to what lies within us. ” Oliver Wendell Holmes

Thank You!

“The intern suffers not only from inexperience, but also from over-experience. He has, in his short term of service, responsibilities which are too great for him; He becomes accustomed to act without preparation and he acquires a confidence in himself and self-complacency which may be useful in times of emergency, but which tends to blind him to his inadequacy and to warp his career. ” William Stewart Halsted, 1904

Issues with “Competency Based” Threshold Decisions for Completion of Training Performance ed r to tice n e ac M d e Pr e nu ativ i t n r Co elibe D Derived from K. Anders Ericsson. Deliberate Practice and the Acquisition and Maintenance of Expert Performance in Medicine and Related Domains. Acad. Med. 79(10), S 70 -S 81, 2004 Expert Performance Continue Deliberative Practice Continue Complex Cognitive/Associative Development Arrested Development Cease Deliberative Practice Automated Performance Everyday Skills Cognitive/Associative Development Terminated Automated Performance EXPERIENCE

Deliberative Practice in the Development of Professional Violinists TJN from K. Anders Ericsson. Deliberate Practice and the Acquisition and Maintenance of Expert Performance in Medicine and Related Domains. Acad. Med. 79(10), S 70 -S 81, 2004

Reliability Across Evaluation Methods Slide derived from that created by Cees van der Vleuten, 2010 as presented by Brian Hodges, ACGME Annual Educational Conference, 3/2/2013 Testing Time (hrs) Multiple Choice Questions 1 Case Based Short Essay 2 Patient Management Problems 1 Oral Examination 3 Long Case 4 Objective Structured Clinical Evaluation (OSCE)5 1 0. 62 0. 68 0. 36 0. 50 0. 60 0. 47 2 0. 76 0. 73 0. 53 0. 69 0. 75 0. 64 4 0. 93 0. 84 0. 69 0. 82 0. 86 0. 78 8 0. 93 0. 82 0. 90 0. 88 Review Article Reference: van der Vleuten, CPM, Schuwirth, LWT. Assessing professional competence: from methods to Programmes. Medical Education 2005; 39: 309– 317 1 Norcini JJ. Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence. Medical education. 1985; 19(5): 238 -47. 2 Stalenhoef-Halling et al. , 1990 3 Swanson, David B. A measurement framework for performance-based tests. In: Hart I, Harden R, eds, Further Developments in Assessing Clinical Competence. Montreal: Can-Heal publications 1987; 13– 45. 4 Wass, V. , Jolly, B. Does observation add to the validity of the long case? Medical Education 2001; 35: 729± 734 5 Petrusa, ER. Status of Standardized Patient Assessment. Teaching and Learning in Medicine, 16(1), 98– 110 and: Clinical performance assessments. In International handbook of research in medical education. Dordrecht, The Netherlands: Kluwer Academic, 2002.

Reliability of Oral Examinations 1 Slide derived from that created by Cees van der Vleuten, 2010 as presented by Brian Hodges, ACGME Annual Educational Conference, 3/2/2013 Testing Number Time (hrs) of Cases Same Examiner for All Cases New Examiner for Each Case Two New Examiners for Each Case 1 2 0. 31 0. 50 0. 61 2 4 0. 47 0. 69 0. 76 4 8 0. 47 0. 82 0. 86 8 12 0. 48 0. 90 0. 93 Swanson, David B. A measurement framework for performance-based tests. In: Hart I, Harden R, eds, Further Developments in Assessing Clinical Competence. Montreal: Can-Heal publications 1987; 13– 45. See also, Swanson, DB. Arch Intern Med 1987; 147: 1981 -1985. 1

Accreditation Council for Graduate Medical Education The CLER Program one element of ACGME’s Next Accreditation System

CLER Focus Areas Patient Safety Supervision Duty Hours Fatigue management Transitions of care Healthcare Quality Healthcare Disparities Professionalism

Clinical Learning Environment Review (CLER) Program • First Cycle • 380+ Sponsoring Institutions with multiple programs • Used solely for feedback, learning, and establishment of baseline information • Identification of salutary practices © 2013 Accreditation Council for Graduate Medical Education (ACGME)

CLER Program 5 key questions for each site visit • Who and what form the hospital/medical center’s infrastructure designed to address the six focus areas? • How integrated is the GME leadership and faculty in hospital/medical center efforts across the six focus areas? • How engaged are the residents and fellows? • How does the hospital/medical center determine the success of its efforts to integrate GME into the six focus areas? • What are the areas the hospital/medical center has identified for improvement? © 2013 Accreditation Council for Graduate Medical Education (ACGME)

SCHEMATIC OF FLOW OF CLER SITE VISIT Resident meeting CPS/CQO meeting Walk. Around II Walk. Around I Core faculty meeting Walk. Around III P. D. meeting Exit meeting DIO, GMEC Chair CEO, CMO/CNO Initial meeting DIO, GMEC Chair, CEO, CMO/CNO Team Prep meeting Foundational Learning Exploration and Inquiry Review, Clarification & Feedback Three phases of Visit Note: each walk around accompanied by resident host/escort, opportunity for staff (e. g. nurses) and patient contact (future). As yet, uncertain of role of interview with governance interview. © 2013 Accreditation Council for Graduate Medical Education (ACGME)

CLER Evaluation Process* 1 CLER Sponsoring Institution Site Visit (Cycle I n=380+) 3 Possible egregious violation Site Visit Oral Report Site Visit Report initial feedback Institutional response (optional ) Copy of report sent back to institution, allow for response 2 CLER Program Staff Preparation for Committee Review (Completeness and attachment of any institutional response) Cycle I: To IRC (aggregate, de-identified) CLER Committee Report (final) Evaluation Committee Review * Approved by CLER Evaluation Committee 10/2012 CLER Process Map Ver 2. 1, 12. 7. 2012

Example of possible template for categorizing CLER engagement Increasing levels of GME engagement with participating site Patient Safety Resident reporting of adverse events Category 1 All residents are aware of and expected to report events Category 2 Some residents are reporting events and near misses Category 3 Category 4 Most residents are reporting events and near misses Education on patient safety Learning environment culture of safety Under development Resident experience with safety investigations © 2013 Accreditation Council for Graduate Medical Education (ACGME) Nearly all residents are reporting events and near misses

CLER Feedback • What they are intended for: • Experiences that inform learning • Guides for voluntary improvement efforts • A progressive set of activities for higher performance in organizational engagement in GME • Basis for empiric understanding of what is possible • Indicate areas ripe for future work • What they are not intended for: • Gotcha’s • New stealth accreditation requirements © 2013 Accreditation Council for Graduate Medical Education (ACGME)

CLER Program: alpha and early beta visits © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

Early Impressions • Experience: • • • >400 residents >300 faculty >200 program directors Hospital leadership Numerous nurses and other care providers • Caveats/disclaimers • 15 visits • Nearly all volunteering institutions • Early use of site visit protocol © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

Early Impressions • Patient Safety • Variability in residents knowledge of when, what and how to report • Healthcare Quality • Degree and experience of resident participation in QI varies across programs • Variable alignment with the clinical site’s priorities • Disparities initiatives focus on access; little attention to measuring variability or impact © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

Early Impressions • Transitions of Care • Primary focus on hand-off for change of duty • Variability in process and oversight of resident hand-offs • Supervision • Examples of both under and over supervision • Knowledge of need for direct supervision appears to be limited to GME faculty © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

Early Impressions • Duty Hours/Fatigue Management • Consistent emphasis on education; variable evidence of effective management strategies • Professionalism • To date, most residents report being in a culture of openness for bringing forth concerns regarding honesty in reporting • Variable monitoring by participating site © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

Some key observations • Learning begins before visit and continues after the site visit • Attempt to “prep” individuals with “right” answers may not work • The concept of ACGME providing formative feedback not judgment is different • In post-visit planning of improvement do not try to “boil the ocean. ” © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

A couple of additional thoughts • Significant variability in participating site’s leadership view of the strategic value of GME in advancing patient safety and care improvement © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

A couple of additional thoughts • Significant variability in participating site’s leadership view of the strategic role of GME in advancing patient safety and care improvement © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

Some of the next steps in development of CLER Site Visits • • • Continue to refine protocol Increase site visit staff Aligning with CLER Evaluation expectations Implement quality management process Test new sub-protocols • • Unique/special S. I. ’s (e. g. Military, VA, Childrens) Patient encounter Contact with governance SI site visits to multiple participating sites © 2013 Accreditation Council for Graduate Medical Education (ACGME) March 20, 2013

Accreditation Council for Graduate Medical Education The CLER Program one element of ACGME’s Next Accreditation System

Entrustable Professional Activity EPA’s • Real life patient care episodes • Usually composed of elements of most if not all “competencies” • Benchmark of performance is the ability to be entrusted to perform care with “indirect supervision with direct supervision available” • Progression is then the achievement of EPA’s of increasing difficulty, risk, or sophistication • Proficiency is then the achievement of the most sophisticated EPA’s required of the resident

The Goal of the Continuum of Professional Development in the 4 year preparation of the Anesthesiologist Master Expert EPA EPA EPA Proficient EPA Competent Advanced Beginner EPA EPA EPA EPA EPA Novice PGY 1 PGY 2 PGY 3 PGY 4 MOC Increase the Accreditation Emphasis on Educational Outcomes