Prepare and Use Skills Assessments Introduction Why do

Prepare and Use Skills Assessments

Introduction • Why do we assess student clinical skills? • What is challenging about preparing and using skills assessments?

Steps in Skill Development • Introduce and demonstrate a skill. • Observe students as they practice the skill and give feedback to help them improve their performance. • Assess students for competency in the skill.

Objectives • • • Select methods for assessing the skills of students Prepare skills assessments Develop structured practical examinations Administer and score skills assessments Use results to improve performance

Skill Assessment Methods • Direct observation of students as they perform skills • Structured feedback reports on students’ performance • Logbooks, learning journals, and care plans

Direct Observation #1 • Most valid way to assess students’ skills. • Can be used for both formative and summative assessments. • Need to be creative developing approaches when you have large numbers of students.

Direct Observation #2 • Oral questioning can be used with direct observation. • Can be done with simulations (formative) and with patients (summative).

Structured Feedback Reports #1 • Assessing sustained performance rather than just taking “a snapshot” as you would with an examination. • Can coverall performance, demonstrated attitudes, and essential healthcare delivery skills. • Useful for assessing characteristics such as personal attributes, attitudes, and professional values.

Structured Feedback Reports #2 • • Are easy, efficient, and consistent Provide a formal structure for assessment, particularly formative assessment Reinforce essential skills Ensure that each student receives feedback • See Sample 11 -1 and 11 -2

Logbook • The logbook (also called a casebook) contains a list of skills or tasks that students should be able to perform. • The students are responsible for learning how to do each of the tasks, and when they believe they are ready, they can ask a teacher, tutor, or clinical instructor to assess their performance.

Learning Journal A learning journal is used to record learning experiences, especially those in which the student has minimal or no supervision, such as home visits, community-based experiences, or rotations to distant clinic sites.

Care Plan • A care plan is used to document the patient’s problems, care required, and expected outcomes. • Students are often required to create care plans to demonstrate their understanding of and ability to explain management required for a specific problem.

Checklists • A list of steps needed to perform a skill correctly, listed in the correct sequence. • Assessor observes each step. • Well-constructed checklists should contain only sufficient detail to help the assessor evaluate and record the student’s performance.

Designing a Checklist #1 • Identify the steps or tasks: • • Adapt an existing tool, or Conduct a task analysis • Place the steps in the correct sequence. • Identify the standards or minimum level of performance (Yes/No or Multi-level rating systems).

Designing a Checklist #2 • Include the key elements of a checklist (title, space for names, course information, instructions, space for signature). • Field-test the checklist. • See Samples 11 -3, 11 -4 and 11 -5.

Structured Practical Examinations #1 • Typically students rotate through a series of stations where they answer questions (orally or in writing), or perform tasks while being observed. • Students may demonstrate a skill, interpret diagnostic materials, or respond to short questions or case studies.

Structured Practical Examinations #2 • This type of examination is also known as a multiple station assessment test (MSAT). • The Objective Structured Clinical Examination (OSCE) is one of the most well known forms of the structured practical examination.

Components of a Structured Practical Examination • All students rotate through multiple stations and are tested on the same KSAs. • There is a time limit for each station. • An assessor at each station that requires observation. • All students are assessed according to the same standards.

Steps to Prepare for a Structured Practical Examination #1 • Choose which learning objectives will be tested. • Decide on a problem, issue, or activity that addresses each learning objective. • Map out a plan for the stations (see Sample 11 -1).

Steps to Prepare for a Structured Practical Examination #2 • Plan the details for each station: • Write the task/scenario to be completed. • Develop instructions. • Develop assessment tools. • List the resources needed.

Before the Skills Assessment • Discuss previous practice sessions with the student. • Ask if the student has any questions about the skill and is ready to be assessed. • Review the assessment tool.

During the Skills Assessment • • Observe and assess the student’s performance: Stand where you can see without intruding and let the student perform the skill. Do not interfere unless the student is about to make a mistake that may endanger the patient. Provide only essential feedback while the student is performing the skill.

After the Skills Assessment • Review the skill with the student (student shares what she or he did well and what could be improved). • Provide positive feedback and offer suggestions for improvement. • Determine if the student is competent or needs additional practice.

Use Results to Improve Performance #1 • Give students an opportunity to ask you questions about steps they did not understand or they performed incorrectly. • Instruct students to practice the steps that they performed incorrectly.

Use Results to Improve Performance #2 If many students had trouble with the same tasks, either the teaching methods or materials did not adequately cover that learning objective, or the task needs to be redefined.

Summary #1 • Direct observation is the most valid method for assessing skills, but can be influenced by the judgment of the observer. • You can improve this method by using standardized tools such as checklists to guide assessment.

Summary #2 • Structured practical examinations require time for planning and preparing valid stations, but provide a highly structured and reliable method for assessing knowledge, skills, and attitudes that can be used from year to year. • Assessing students’ skills is one of the keys to improving students’ performance.

Discussion • In the large group or in small groups: • What is the role of the EDC, PIC and QCC in assisting with student knowledge and skills assessment? • What challenges do we face in strengthening student knowledge and skills assessments?