Understanding the Accreditation Reform Presentation Overview 1 Accreditation

Understanding the Accreditation Reform

Presentation Overview 1. Accreditation Reform: Overview 2. New Accreditation Standards and Processes 3. Progressive, Eight Year Accreditation Cycle 4. AMS Platform: A New Digital System 5. Emphasis on Self-Study and CQI 6. Onsight Visits Moving Forward 7. Next Steps: What to Expect

Accreditation Reform: Overview Understanding Accreditation, Can. RAC, and the Reform

What is accreditation? • Quality improvement process • Safeguards adherence to national standards • Ensures all graduates are ready for practice

What is the Can. RAC?

Background: Initiation of change process • No system-wide review in many years • Conjoint Accreditation Taskforce 20112012 – Interviews with PG Deans – Challenges identified & Recommendations made for a reformed system – 30% of PG Deans called for transformative change 6

How did we get here? High-level timeline • • 2011 -2012: Royal College struck an accreditation reform committee 2012: Focus groups with PG Deans and other stakeholders confirm the need for change 2013: Three colleges partner to evaluate the current accreditation system, start brainstorming. 2014: Three colleges begin developing/consulting on a plan to introduce a new system 2014 -2015: Begin to release elements of the proposed plan Fall 2015 -Spring 2017: Prototype 1 begins (Laval, Saskatchewan, Sherbrooke, Memorial and Ottawa) February 2016: All 3 colleges receive endorsement (in principal) from the accreditation councils

Why change the system? Proposed reform elements were built on stakeholder feedback, and aim to: • Create new general standards that provide clarity, reflect updated Can. MEDS Framework, and supports transition to CBME practices. • Reduce burden of work, promotes continuous evaluation and quality improvement, and integrates innovative practices. • Digitize the accreditation system. 12/5/2020 8

What’s changing? What’s not? Old System Proposed Systematic rigorous process Peer review × Too much paperwork Digital platform × High stakes, “snap-shot in time” Continuous cycle of site visit accreditation × Lack of focus on QI outcomes Increased focus on outcomes, competency-based programs × Inappropriate categories of accreditation Revised categories × Resident input not optimized Robust system for resident involvement × Idiosyncratic nature of decision- Clearer standards, standardized making and reproducible decisions × High stakes, punitive Emphasis on continuous improvement

New Accreditation Standards and Processes Understanding what the changes are, and how the overall system will benefit

Understanding the proposed reform components Proposed Conjoint Accreditation System Reform Components 1. New Standards for programs and institutions. 2. A new evaluation framework of standards for residency programs, including Exemplary ratings and best practices. 3. A new progressive accreditation cycle of regular accreditation visits, supported by continuous data monitoring. 4. Introduction of a digital Accreditation Management System. 5. Increased emphasis on self-study and continuous quality improvement. 12/5/2020 11

Understanding the proposed reform components 6. Enhanced onsite review processes, such as tracer methods. 7. A new institutional review process, standard system, and status category. 8. A renewed emphasis on the quality and safety of learning environments. 9. New decision categories, with thresholds to improve consistency of decision-making. 10. A systematic approach to evaluation, research, and continuous improvement of the system. 12/5/2020 12

Benefits of the new proposed standards • The new proposed general standards will: – Provide greater clarity, without being overly prescriptive. – Reflect the new content of the Can. MEDS 2015 Framework. – Support the transition to competency-based medical education. – Place greater emphasis on the learning environment. 12/5/2020 13

Standards Domain Definition Institution Level Institutional Governance Standards that relate to the overall oversight of medical education at the institutional level and governance of the educational mission. √ Program Organization Structural and functional standards related to the administration of the education program. Education Program Level Example Markers & Evidence e. g. support for education (promotion policies) √ e. g. PD protected time (interviews w/ PD, others) Standards related to the design of the education program, its goals/objectives, the specific content required in the academic curriculum. , and the assessment of learners and their readiness for practice (assessment and achievement of competencies). √ e. g. comprehensive plan for teaching and assessment (curriculum map or blueprint) Resources Standards include those dedicated to sufficiency of ALL resources (both education program specific and broader resource issues). √ e. g. patient/ procedural volumes (e. Log, e. Portfolio) Learners, Teachers & Administrative Personnel Standards relevant to the people most directly involved in the delivery of residency education, namely teachers, learners, and administrative personnel. √ e. g. learning environment that protects patient, resident and faculty safety (learner survey) Continuous Improvement Standards relate to ensuring the program/ institution has effective continuous improvement mechanisms and processes √ √ √ e. g. institution & program involvement in CQI

Why are the processes changing? • The proposed changes aim to: – Reduce the burden of work on schools. – Promote continuous evaluation and quality improvement. – Integrate innovative practices into the accreditation system. 12/5/2020 15

Progressive Accreditation Cycle

Why change from a six to eight year cycle? Current challenges Future improvements • Overemphasis on high stakes, “snapshot in time” onsite • Enhance emphasis on • Episodic nature of accreditation – High burden preparation for accreditation visits accreditation as CQI • More continuous accreditation cycle supported by ongoing data monitoring

What will an 8 year cycle look like? • 8 years between regular accreditation visits – With predictable 2 year follow-ups • Introduction of data collection from variety of sources to enhance evaluation of clinical learning environment – Aggregate data (e. g. , survey, etc) intended to contextualize program quality and safety – Multiple sources of aggregate including graduates, residents, teachers, others

Continuous Data Monitoring in the New Accreditation Cycle 2020 2021 2022 2023 (Base Year) Nov (Year 2) Nov AI with follow-up by RS ONSITE SURVEY APOR or ER APOR/ER of Programs (as required) Institutional Review 2024 (Year 4) INTERNAL (mid-cycle) AND APOR or ER Expectation that AI conducts the program IRs APOR/ER of Programs (as required) 2024 2025 2026 Nov (Year 6) 2026 2027 Nov APOR or ER APOR/ER of Programs (as required) 2028 (Year 8) ONSITE SURVEY Institutional Review Program Reviews CQI & data monitoring LEGEND: AI = Accredited Institution Rectangles = Accreditation Activity Ovals = Accreditation Status Purple = Institution Level Blue = Program Level RS = Regular Survey APOR = Action Plan Outcomes Report ER = External Review IR = Internal Review ONGOING DATA MONITORING: SURVEYS (graduates, residents, faculty) and MILESTONES → deviation from mean = trigger for accreditation activity (e. g. , progress report, external review) ROBUST PROCESS TO IDENTIFY SIGNIFICANT PROGRAM CHANGES → trigger for accreditation activity (e. g. , progress report, application)

AMS Platform: A New Digital System Understanding the new Accreditation Management System (AMS)

Recommendations for change Current challenges • PSQ: administrative burden – High intensity, timecondensed work – Used episodically – Not reliable or easy to use for CQI Future improvements • Digital AMS – Single repository – Integration with other systems – Intuitive, userfriendly – Guides CQI activities

Proposed AMS functionality An Accreditation Management System (AMS) equipped with: Program portfolio Self-Study Tool Action Plan

Program Portfolio • Houses all accreditation information (e. g. documentation, policies, etc. ) • Directly linked to new standards • Customized notifications • Prepared at all times for internal and mandated CQI activities • Guidance tool available

Self-study and CQI Current challenges • Limited selfevaluation within PSQ • Accreditation as high stakes, sometimes punitive Future improvements • Self-Study tool – Mimics onsite process – Could be used for IR – Easily identify strengths, AFIs at any time • Accreditation aligned with principles of ongoing CQI

Self-Study Tool • Easy to use • Directly linked to new standards • Easy access to required sources: – “Evergreen” program profile (documents, policies, etc. ) – Information/Data from external sources (e. g. faculty survey, aggregate data from e. Portfolio or equivalent, such as “Time Stamps”). • Facilitates alignment to new Continuous Improvement standards

Onsite Visits Moving Forward Program and Institutional Reviews

Recommendations for change Current challenges Future improvements • Time-condensed, high volume prep • Continuous access and updating of info in AMS • Lots of paper work • Personalized prompts • Does not support continuity of workflow for accreditation activities • Submit information online via “Publish” functionality

Onsite Visits: What stays the same? • Peer review • Experiencing an onsite visit • Scheduled meetings with key stakeholders • Documents/systems made available to surveyors • Identification of program strengths and areas for improvement

Onsite Visits: What will be new? • All information online • Flexibility in the review schedule • Interview guides to facilitate surveyors’ work • Conduct tracers mapped to standards • Identify and facilitate sharing of innovative and best practices between programs AMS

Next Steps What to expect in the years ahead

What’s Next Through 2016? • National Consultations – Key principles and implementation • (Spring/Summer 2016) – Standards and detailed process elements – wider audience • (fall 2016) • Preparation for final approvals via all three college’s AC (2017) and phased implementation (2017 and beyond)

32

We’re here to help! We know that transitioning to a new accreditation system is complex. If you have questions/feedback about the reform, we encourage you to reach out to us at: info@canrac. ca 33