Review of Year 3 ICE course Course occurs

Review of Year 3 ICE course • Course occurs in 3 sessions, 1 -3 days each, June or July/January/June of Year 3 • Course Director – John Dick III, MD • Course has ~30 curricular hours • Course was last reviewed in 8/2015 • Current review 8/2017

Action Plan from Prior Review • Objectives 1. 2. 3. 4. 5. # 2 - Eliminate words “disease prevention, risk factor modification” as these are not taught in this course. • Done Modify and combine objectives # 6 and # 7 1. 6. Demonstrate performing common medical procedures in a simulation setting. 2. 7. Explain the indications, complications, limitations, and performance of common tests and procedures. • Perform common medical procedures in a simulation setting and explain their indications, complications, and limitations. – Done, however subsequently dropped from course Add a cultural competence objective • Recognize and address cultural biases in yourself and others - Done Reword #14 to “Access medical library resources from remote site. ” 1. Done Reword #15 to better emphasize IPE aspect 1. 15. Identify appropriate resources to support patient care and to collaborate effectively with all members of the inter-professional team. • Replace 15 with: Collaborate effectively with all members of the inter-professional team. - Done

Action Plan from Prior Review • Pedagogy – More efficiently coordinate “Delivering Bad News” with Surgery clerkship and Palliative Care ICE session • Done – Better manage SW participant at Health Care Team panel • Done – Minimize Student Affairs session at Mid-year session • Done – Increase near-peer involvement • Will continue to expand as schedules permit – Condense end of year sessions • Done – Rework Pain session • Continue to modify – Decrease note writing time for final OSCE • Decreased from 15 min to 10 min in final OSCE

Action Plan • Assessment – Add more post-session quizzes to help students prioritize and consolidate learning • Trialed this, unpopular and not sure it added value, will reattempt

Course Objectives Course Objective Learning Activity • course objectives listed on this slide Learning Assessment 1. Apply current medical and clinical knowledge to OSCE; Pain Session; Palliative diagnosis and treatment issues in patient care. Care OSCE 2. Apply current knowledge of end-of-life and palliative care, pain management, medical ethics, Pain, Ethics, Palliative Care, and medical-legal issues to clinical problems. HIPAA, Risk Palliative SP, Attendance at session, Ethics write up 3. Interview patients skillfully, utilizing a focused OSCE history. 4. Examine patients skillfully and respectfully, with appropriate attention to student infection control OSCE and patient comfort and privacy. 5. Define and prioritize the patient's problems accurately and generate an appropriate differential diagnosis. 6. Perform common medical procedures in a simulation setting and explain their indications, complications, and limitations. 7. Recognize and address cultural biases in yourself and others. 8. Communicate effectively with patients and families. OSCE Needs activity Needs assessment OSCE; Palliative Care OSCE

Course Objectives • course objectives listed on this slide 9. Communicate effectively and collegially with physician colleagues and other members of the health-care team verbally, in writing and in the electronic medical record. 10. Meet professional responsibilities fully. OSCE, EMR session OSCE Professionalism Session None specific - attendance at session 11. Adhere to professional standards including high Professionalism Session, HIPAA ethical codes, accept responsibility for personal Ethics write up actions, and respect patient confidentiality. Session, Risk Management 12. Demonstrate responsibility for one's own mental and physical health. Resilience Session None specific - attendance at session 13. Demonstrate responsibility for one's own medical education, and develop the habits of mindfulness and reflection. Orientation, Resilience Session, Residency Advising Ethics Write Up Library Session None specific-attendance at session 14. Access medical library resources from remote sites. 15. Collaborate effectively with all members of the Interprofessional Session inter-professional team. None specific-attendance at session

Mapping of Course Objectives to Geisel Competencies • Course director will work to map objectives once they are approved.

Course Objectives – Comments • Is the number of objectives appropriate? (typically 8 -15 – not too broad, but not too detailed). – OSCE exercise bring together many competencies, hence the larger number for such a short course • Do the course objectives encapsulate the main ideas of the course, i. e. when you read the objectives, do you have a good idea regarding what the course is about? – Does not cover orientation materials or residency advising which uses a lot of the available time. • Objectives written clearly and appropriately high up Bloom’s taxonomy? – Yes

Format of Course & Session Objectives • Course objectives are provided in the syllabus – Yes • Course objectives are written in the correct format – Yes • Session objectives are provided in the course materials – Yes • Session objectives are written in the correct format – Yes

Issues of Redundancy • Are there major issues of redundancy with other courses? – Ethics, e. DH training (online vs discussion)

Look back for preparation on key concepts • Are there major issues that should be included with other courses? – EHR – Cultural Competency • Implicit bias? – Procedures • If not in this course, then assure in clerkships (NGT, Foley)

Health and Values Goals Ethics – “Identify key concepts in health care ethics and demonstrate an ability to recognize ethical issues arising in patient care and population health and to think critically and systematically in applying an ethical analysis” Cultural Awareness – “Demonstrate an understanding and skill in managing patient care of people of diverse cultures, social, economic standing and belief systems” Health Equity – “Identify the root causes and approaches for addressing health disparities locally and globally” Resilience – Demonstrate knowledge of skills and practices to prevent and address stress and maintain resilience in caring for patients and oneself Compassion and Empathy – “Demonstrate abilities to understand each patient’s experience of illness, adapt scientifically appropriate care to conform to that patient’s needs, and communicate in terms that each patient can understand” There also are synergies to health law, communication skills, professionalism (as LCME requires).

Health and Values Content • • • What Health and Values Program material (healthcare ethics, cultural awareness, health equity, resilience, compassionate care) is presented in the course? – Applied Ethics – 2 sessions and write up – Resilience Session – Compassionate Care – Palliative Care Sessions – Cultural awareness Are the Health and Values topics noted in the course and session objectives? – Yes, Ethics and resiliency, cultural awareness What do the student evaluations indicate regarding Health and Values teaching? – Valued experiences

Summary regarding Objectives • Overall objectives appropriate with the following exceptions: – Dropped Procedures session from course – MEC will need to consider if and where these belong – Redundant Professionalism objectives (10 and 11) • Health and Values adequately covered • No significant unplanned redundancy with other courses

Course Learning Opportunities • Lecture/Large Group Discussion 20 hrs. • Panel 3 hrs. – Health Care Team – Student to Student Advising • Case based large group 3. 5 hrs. – Pain/Opioid Session – Ethics • Simulation 2 hrs. – Two 30 min OSCEs – One small group responding to emotion SP with feedback

Course Learning Opportunities • Timing of course (just prior to first clerkship, post vacation and immediately post Year 3) a challenge in terms of pre-reading and engagement • Mixed pedagogy but still heavy on lecture/large group based activities – Moved to decrease the latter by using DH e. Learning modules for HIPAA, RISK, COMPLIANCE, ENVIRONMENTAL SAFETY • OSCEs popular and recommend more per student

Summary regarding Pedagogy • Highlights include OSCEs, more of which are recommended and being worked upon. • Advising and some didactics still lecture based • More small groups would be helpful – consider bringing back together On-Doctoring groups with facilitators for certain topics

Assessment • Ethics Write Up: P/F • OSCE x 2: – Scored on Patient interview/exam and SOAP note; less than 2 SD below the mean requires meeting with course director for in person review and potential repeat OSCE – More detailed feedback requested and being worked on • Post session quizzes in the past and encouraged (low stakes, 1 to 2 key points) again

Assessment for Course Objectives • Not every course objective assessed individually – – Library resources Cultural Biases Personal Health Collaborate effectively with health care team • Consider short post session quizzes for these

Summary regarding Assessment • OSCEs – More detailed feedback requested • Course objectives – More post session quizzes

Measures of Quality – Step II CS Geisel Pass rate vs National 2012 2013 2014 2015 2016 2017 97% / 98% 99%/ 98% 100% / 97% 100%/95% 98/96% 97%/95%

Measures of Quality – Course Evaluation scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent] IIncoming ICE 2012 2013 2014 2015 2016 Overall satisfaction of course 3. 15 n/a 3. 74 3. 88 Welcome/Overview 4. 58 Nuts/Bolts 4. 32 Radiology 3. 63 n/a 4. 16 n/a Student Affairs 3. 93 4. 01 4. 24 4. 15 Procedures 4. 61 4. 28 4. 42 4. 47 n/a Health Care Team 2. 55 3. 74 n/a 3. 98 4. 26 Ethics 3. 57 3. 71 3. 87 n/a Risk 4. 03 4. 07 3. 79 n/a 4. 4 Environmental Safety 3. 75 3. 67 3. 94 3. 75 3. 91 EMR 3. 66 3. 36 n/a 3. 86 4. 23 Interpreter Services n/a 3. 79 n/a 3. 45 Resiliency n/a n/a 4. 15 4. 56

Measures of Quality – Course Evaluation scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent] Mid-Year ICE 2012 2013 2014 2015 2016 Student Affairs 3. 71 4. 06 3. 59 4. 0 3. 71 Usefulness of OSCE 3. 72 3. 65 4. 51 4. 46 4. 18 3. 73 (death/d ying) n/a 4. 25 3. 88 3. 98 Pain 3. 56 n/a 3. 52 3. 92 3. 78 Residency Advising 4. 42 4. 40 4. 81 4. 82 4. 7 Year 4 Intro 4. 46 4. 43 4. 63 Cultural Competence n/a 3. 59 Palliative Care 4. 58 n/a Informatics 3. 7 n/a Social Media 3. 82 n/a Ethics 4. 19 3. 94 Resiliency 4. 07 n/a

Measures of Quality – Course Evaluation scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent] Outgoing ICE 2012 2013 2014 2015 2016 Overall satisfaction of course n/a 3. 71 (3. 94 overall) 3. 91 3. 92 (3. 89 for End of Year) Usefulness of OSCE n/a 3. 81 4. 30 4. 25 (w/OSCE) 4. 23 (w/OSCE) Palliative Care n/a 3. 72 4. 03 4. 25 (w/OSCE) 4. 23 (w/OSCE) Medical Ethics 4. 07 3. 66 Step 2 prep 4. 27 4. 23 Residency Advising 4. 47 4. 43 4. 47 4. 52 4. 26 Student Affairs n/a 3. 89 3. 88 3. 87 Medical Translators n/a 3. 05 n/a n/a Sub. I Primer/Post Clerkship Skills 4. 01 4. 19 Health Equity 3. 81 n/a

Measures of Quality – Student Comments ICE - Incoming Year 3 Overview - Students appreciate that orientation has been condensed down to just the essentials - “I really appreciated that our time was valued and every speaker/session was relevant and made good use of time” - Students were particularly complimentary of Dr. Dick and Dr. Duncan’s sessions. - Consider further condensing down to 2 -days - Some felt that the sessions were not applicable enough to the medical student experience, but rather practicing medicine after residency - Many commented on the need for in-person e. DH training as well as not requiring e. DH training to be completed during STEP 1 study time - Some wanted more ‘nuts and bolts’ training sessions such as including a hospital tour. Risk Management - A few students pointed out what they heard as “offensive” language during the session and mentioned that the speaker was unprofessional. Lunch with 4 th Years - Many compliments on the usefulness of the panel - Some suggested the panel go on for longer and faculty step out of the room sooner - Suggested that a Q&A document be compiled of the session for future reference

Measures of Quality – Student Comments ICE - Incoming Year 3 (cont. ) Environmental Safety - Students understand the necessity for the session, but question the length. Respiratory Fit - Notify men the shave the day before. - Consider methods to decrease down time during the session Health Care Team - Session was overall very well received - Suggested inclusion of OR/scrub nurses in discussion Medical Informatics - Felt session was redundant after required online e. DH training Resiliency - Very well received session - “CHANGED MY LIFE” Medical Interpreters - Multiple complaints throughout the evaluation on the utility, length and quality of the session. - Suggestion of making this an online session or incorporate training in using the interpreter services at DHMC

Measures of Quality – Student Comments ICE - Mid-Year Overview - Overall complimentary especially in regards to the condensed nature of the day Residency Advising - Students were very complimentary of Dr Harper - Suggested creating a formal timeline for material Responding to Emotion - Session was very well received - Students offered the idea of presenting this material in the earlier session or incorporating it into On-Doctoring Student Affairs Update - Students felt it was repetitive and could be covered in an email update Medical Ethics - Students were complimentary of the speakers, but felt the session was exactly the same as previous ethics sessions throughout year 1 and 2. Pain Management - Students enjoyed Dr Dent, but believed the session assumed a certain level of knowledge and found the presentation difficult to follow - Suggested shortening session to 30 mins OSCEs - Very helpful as a training tool for STEP 2 CS - Suggested adding oral presentation for purposes of getting constructive feedback for rotations - Suggested allowing for OSCEs to be done before break.

Measures of Quality – Student Comments ICE - Outgoing Overview - Felt the sessions were fun and concise - Student were happy to see the staff, but felt the session was redundant Student Affairs Updates Post Clerkship Skills - Overall students found this to be a very helpful session Intro/Step II CK/CS Tips - Students would have like to have gotten some resources to reference (ie study schedules, materials used, student panel, etc. ) Applied Medical Ethics - Students felt the two speakers were incongruent with their intentions for the session - People enjoyed the small group discussion aspect - Some expressed frustration about having to complete an assignment during another rotation and were critical of the format of the assignment in general - Suggested breaking up into year 1 and 2 on-doctoring groups and removing or altering assignment Residency Advising - Some students found the session useful, but overwhelming and stressful OSCE - Consider consolidating OSCE/Palliative Care session into one time slot - Suggested performing 2 OSCEs back to back for STEP 2 CS practice

Summary regarding Measures of Quality • Generally well-received course with positive student feedback. • OSCEs felt to be good preparation for Step 2 CS and students do well

Recommendations • Objectives – Remove # 6 “Perform common medical procedures in a simulation setting and explain their indications, complications, and limitations. ” but discuss where these might better belong. – Remove # 10 “Meet professional responsibilities fully. ” and alter # 11 to include “professional”: Adhere to professional standards including high ethical codes, accept responsibility for personal actions, and respect patient confidentiality. – Change #14 from “Access medical library resources from remote sites” to ”Use information technology respectfully and responsibly. ” – Change #15 from “Collaborate effectively with all members of the interprofessional team” to “Recognize the different roles and strengths of members of the inter-professional team. ” – Add new Objective to cover the advising that occurs • “Explain how to optimize your clinical learning and residency preparation. ”

Recommendations • Pedagogy – Increase number of OSCEs – Increase small group activities perhaps with reconvening On-Doctoring small groups/facilitators – Rework e. DH and Interpreter Training sessions – Create more specialty specific timelines for residency advising session – Alert students to pending ethics assignment earlier in the academic year • Assessment – Provider more detailed feedback on OSCE – Reintroduce low stakes post session quizzes to highlight key points

Action Plan • • • Objectives – Accept changes recommended by the subcommittee – Discuss where procedural training best fits in the overall curriculum with the Clerkship Directors and MEC Pedagogy – Plan and implement a “phone call” OSCE to increase # and mimic Step 2 CS – Replace interpreter session with e. Learning module – Explore small group session with On-Doctoring groups – Introduce implicit bias training (Harvard on-line modules) to cover cultural awareness – Will create more specialty specific timelines for residency advising – Re-emphasize ethics assignment earlier in year Assessment – Redesign feedback provided to students on OSCE performance – Reintroduce post session quizzes (low stakes, key points)