Clinical Competency Committee Meshing Milestones with Evaluations Laura

Clinical Competency Committee: Meshing Milestones with Evaluations Laura Hempstead, DO, FACOFP STFM 4/27/2015

Disclosures, Goals and Objectives • Nothing to disclose • Identify strategies to overcome initial barriers to starting a CCC • Discuss ways to use a core competency of the block daily feedback evaluation • Utilize alarm bells to identify underperforming residents early and intervene • Develop a global behaviorally based evaluation linking milestones to the evaluation.

Clinical Competency Committee Mock Orals Nursing and Ancillary Personnel Evaluations Procedural Evaluations OSCE Peer Evaluations Self Evaluations End-of. Rotation Evaluations ITE Sim Lab Case Logs Unsolicited Comments Student Evaluations Clinical Competency Committee Assessment of Milestones Clinic Workplace Evaluations Patient/ Family Evaluations

Who should be members of CCC • Program director/associate PD • Core faculty from IM, OB, family medicine center, other key rotations • Residency coordinator • Clinic manager/PCMH manager • Behaviorist and/or social worker • Invite advisors when their residents are being discussed • Input from chief residents and RICs crucial

The milestones are overwhelming, how do we get started? • Faculty education on content of milestones • Resident education on content of milestones • Quick-reference sheets provided for all faculty and residents • Alarm bell reference sheets provided for all faculty and residents

More on Milestones: Defining them in ways pertinent to your curriculum • Set aside a full half day (you may need 2) • Tape the individual milestones to the walls/place on tables around a large room • Provide 2 -3 colors of sticky notes to each meeting attendee (include all interested residents/faculty and support staff) • Each attendee writes their suggestions for behaviors on one color, suggestions for curriculum improvement in another color, fellowship suggestions in a 3 rd color.

Milestone Template

CCC Agenda Template • Week 1 – review multirotor evaluations on the 7 residents being evaluated for the block. • Week 2 – review core competency of the block daily feedback evaluations on the 7 residents being evaluated for the block • Week 3 – OB resident in charge (RIC) and faculty present at beginning of meeting - strengths and areas for improvement of each OB team member • Week 4 – IM RIC and faculty present strengths and areas for improvement of each IM team member • Every week/additional meetings if needed – milestone evaluations for the 7 residents of the block, discuss residents with IEPs not fully addressed during weekly meetings. Monitor success of evaluation tools. Implement new evaluation processes and tools if indicated.

Evaluations used at UMKC Family Medicine Residency Program • General rotation Evaluation – behaviorally based, used for all rotations (Faculty to resident, peer to peer) • Multi-rater evaluation (nursing, non-physician staff) • Alarm Bells evaluation – (on the fly, not mandatory) • Daily outpatient feedback evaluation • RIC evaluation (Faculty to RIC, Resident to RIC) • Quarterly self-evaluation • Portfolio (Test scores, longitudinal projects etc. )

Core Competency of the Block Evaluation Medical Knowledge

Alarm Bells resource

Resident Quarterly Self-assessment • Free Text document • Addresses between 1 and 3 of the milestone behaviors for each core competency • Most of these behaviors are not readily observable by faculty • Completed after 1 st and 3 rd quarters of each academic year.

Resident Quarterly Self-assessment Medical Knowledge: MK-1 • Describe your plan of study to improve medical knowledge: Address how you use the In-Service exam to guide areas for further reading and study. Include comments about your daily study strategy. Patient Care : PC 3 • Describe how you utilize PCMH principles in your patient care and your goals for improvement. Examples: motivational interviewing, pre-visit planning, setting goals with patients, shared decision making, using community resources etc. Interpersonal skills and Communication: C-1, C-2, C-3, P-4 • Provide an example of how you manage stress. Comment on your ability to remain calm in stressful situations. Comment on your stress management goals. • Describe how you give non-judgmental feedback to patients, staff and peers. Lists goals for improvement.

Resident Quarterly Self-assessment Professionalism: P-2, P-4 • Describe how you maintain work-life balance and utilize wellness principles in your own life. • Describe how you exhibit professionalism including a comment on your timeliness, accountability and sense of responsibility. List specific goals you for improvement. Practice Based Learning and Improvement: PBLI-2, P-5, C-4 • Describe your plans for future electives and how these electives will benefit your future practice. • Describe the apps and resources you use to assist with patient care. Describe how you use the library resources. System Based Practice and Improvement: SBP-2, PBLI-1 • Describe how you report medical errors. Discuss how you participate in M&M to learn how to improve our system. List system improvement areas you are working on. • Discuss how you use protocols and evidence-based guidelines to improve patient safety and transitions of care. Outline your plans to improve in this area. • Describe your QI project/Research Project – (PGY 1 and 2 describe topics of interest and how you are acquiring the tools necessary to complete a research project. )

Global Rotation Evaluations • Behaviorally based • Questions limited to 12 • Requires faculty training - Has the option to rate each area evaluated as appropriate for PGY level (globally) or evaluate specific milestones • Doesn’t link to every milestone. (These are addressed using Core Competency of the block evaluations and resident self-evaluations. )

Overall Rotation Evaluation expectations • Level 0 – Alarm Bell Behavior. Resident needs continuous supervision, probation or improvement plan. Some behaviors could trigger immediate suspension. (Comments required) • Level 1 – Level of medical student/new intern in first 6 months of training. If upper level resident is performing at this level, recommend improvement plan in comments. • Level 2 – Progressing PGY 1 or PGY 2 level • Level 3 – Progressing PGY 3 level. • Level 4 – Resident has achieved target milestone for independent practice • Level 5 - Highly accomplished physician, mentor and leader (Comments required) • N/A –If you did not observe this behavior during your rotation.

Rapport: Establishes an ongoing relationship with patients that inspires trust and confidence, elicits cultural factors that impact patient health. (PC-2, PC-4, SPB-4, C-1, P-3) • 0 -Unable to provide clear verbal explanations to patients or health care team, unable to work effectively with patient and family members. Complains and projects negative attitudes with patients and families. Poor interpersonal skills promote lack of confidence. • 1 -Communicates clearly with patients at appropriate literacy level and treats patients respectfully. Recognizes cultural impact on health. • 2 -Creates and sustains nurturing relationships with patients and families, uses minimal jargon and considers patient’s literacy level. Provides consistent patient education and considers cultural factors. • 3 - Creates collaborate relationships with patients which foster patient self-management and trust. Incorporates patient’s beliefs and cultural practices into health care goal setting with patients. • 4 -Creates nurturing relationships with difficult and complex patients, and those with cultural barriers to care. Partners well with complex patients to meet their health care needs. • 5 -Is a leader in cultural proficiency. Develops organizational policies and educational resources to support patient self-management skills and health care goal setting.

Faculty response to new evaluations • Old habits die hard! • Alarm bells utilized more initially, faculty are reverting to face-to-face conversation to express concerns • Core competency of the block adopted by several faculty – not used regularly by others • Global rotation has received positive feedback. Shorter than our prior evaluations. There was some initial confusion. Comments section utilized more formatively after training.

Please evaluate this session at: stfm. org/sessionevaluation

References • http: //acgme. org/acgmeweb/Portals/0/PDFs/ Milestones/Family. Medicine. Milestones. pdf • Swing SR. The ACGME outcome project: Retrospective and prospective. Med Teach. 2007; 29: 648– 654 • http: //journals. lww. com/academicmedicine/Fu lltext/2015/04000/Competencies, _Milestones, _and_Entrustable. 1. aspx