Measurement Measurement Objectives Describe the importance of measurement
Measurement
Measurement Objectives § Describe the importance of measurement § Describe the Kirkpatrick model of training evaluation § Identify measures that can be used to assess the impact of Team. STEPPS § Describe the AHRQ Surveys on Patient Safety Culture § Prepare a plan for determining if Team. STEPPS worked Mod 10 2. 0 Page 2 TEAMSTEPPS 05. 2
Measurement Team. STEPPS Phases Requires Measurement Mod 10 2. 0 Page 3 TEAMSTEPPS 05. 2
Measurement How to Measure - Kirkpatrick Level IV – Results Level III – Behavior Level II – Learning Level I – Reactions Mod 10 2. 0 Page 4 M U L T I L E V E L A P P R O A C H TEAMSTEPPS 05. 2 (Organizational results) (Transfer to the job) (Think, Do, Feel) (Like it and Useful)
Measurement Available Measures n Reactions n n Behavior Course Evaluation Form n n n Learning n n Teamwork Attitudes Questionnaire (T-TAQ) n Learning Benchmarks Team Performance Observation Tool Teamwork Perceptions Questionnaire (T-TPQ) n n Mod 10 2. 0 Page 5 TEAMSTEPPS 05. 2 Team Performance Observation Tool Teamwork Perceptions Questionnaire (T-TPQ) AHRQ Surveys on Patient Safety Culture n Results n n n Patient outcome and clinical process measures AHRQ Surveys on Patient Safety Culture Patient Safety Indicators
Measurement Course Evaluation Form n Level I Reactions n Found in Tab F n Includes evaluation questions for all available course modules n Customizable n Provides information about what trainees thought about the training Mod 10 2. 0 Page 6 TEAMSTEPPS 05. 2
Measurement Team. STEPPS Teamwork Attitudes Questionnaire (T-TAQ) n Level II Learning n Found in Tab F n 30 -item self-report tool n Respondents rate their agreement with items on a 5 -point Likert scale n Measures attitudes toward: n Team Structure n Leadership n Situation Monitoring n Mutual Support n Communication Team. STEPPS Materials and Tools Mod 10 2. 0 Page 7 TEAMSTEPPS 05. 2
Measurement Learning Benchmarks n Level II Learning n Found in Tab F n A 23 -item multiple-choice test n Uses: n To assess knowledge of teamwork n To measure changes in knowledge n Caution n Mod 10 2. 0 Page 8 Items tend to be too easy TEAMSTEPPS 05. 2
Measurement Team Performance Observation Tool n Level II Learning and Level III Behavior n Found in Tab F n Tool for observing team performance n Site assessment n Measure training effectiveness n Observers should practice using the tool n Can be adapted to a particular unit Mod 10 2. 0 Page 9 TEAMSTEPPS 05. 2
Measurement Team. STEPPS Teamwork Perceptions Questionnaire (T-TPQ) n Level II Learning and Level III Behavior n Found in Tab F n 35 -item self-report tool n Respondents rate their agreement with items on a 5 -point Likert scale n Measures staff perceptions of: n Team Structure n Leadership n Situation Monitoring n Mutual Support n Communication Team. STEPPS Materials and Tools Mod 10 2. 0 Page 10 TEAMSTEPPS 05. 2
Measurement Level III: Behavior n Measure whether information learned during training is transferred to the job n Two important factors in transfer are: n Whethere is an opportunity to use the new Team. STEPPS tools or strategies on the job n Whether use of the Team. STEPPS tools and strategies is valued and reinforced Mod 10 2. 0 Page 11 TEAMSTEPPS 05. 2
Measurement Behavior: Measures n Team Performance Observation Tool n Teamwork Perceptions Questionnaire (T-TPQ) n AHRQ Surveys on Patient Safety Culture Mod 10 2. 0 Page 12 TEAMSTEPPS 05. 2
Measurement Level IV: Results n Patient Outcome Measures n Examples: Complication rates, infection rates, measurable medication errors, and patient perceptions of care and satisfaction with their care n Clinical Process Measures n Mod 10 2. 0 Page 13 Examples: Length of patient wait time, time to intubate, medication administration delays, compliance with preventive screenings, number of misdiagnoses, number of structured handoffs used TEAMSTEPPS 05. 2
Measurement AHRQ Surveys on Patient Safety Culture • Free, easy-to-use surveys to assess patient safety culture • Available for various contexts: n Hospital n Medical Office n Nursing Home n Pharmacy • Available on the AHRQ Website Mod 10 2. 0 Page 14 TEAMSTEPPS 05. 2
Measurement Hospital Survey on Patient Safety Culture n Level III Behavior and Level IV Results n Composed of 51 items and 12 composites n Comprehensive instrument that assesses staff perceptions of patient safety culture n Hospitalwide n Specific work area or unit n Comparative database n Mod 10 2. 0 Page 15 Voluntarily submitted data from U. S. hospitals TEAMSTEPPS 05. 2
Measurement How Can Hospitals Use the Survey on Patient Safety Culture? n Raise awareness about patient safety issues n Assess patient safety culture n Track changes in patient safety culture over time n Evaluate the impact of patient safety interventions (e. g. , Team. STEPPS) Mod 10 2. 0 Page 16 TEAMSTEPPS 05. 2
Measurement Survey Measures: 12 Areas of Patient Safety and Two Overall Items Unit-level safety areas • Overall perceptions of safety • Frequency of events reported • Supervisor/manager expectations and actions promoting safety • Organizational learning— continuous improvement • Teamwork within units • Communication and openness • Feedback and communication about error • Nonpunitive responses to error • Staffing Mod 10 2. 0 Page 17 TEAMSTEPPS 05. 2 Hospitalwide safety areas • Hospital management support for patient safety • Teamwork across hospital units • Hospital handoffs and transitions Overall items • The patient safety “grade” respondents would assign their work area/unit • The number of events the respondent reported in the last 12 months
Measurement What Support Is Available? n Hospital Survey Toolkit Survey forms n Survey items and dimensions n Survey User’s Guide n Survey feedback report template n Data entry and analysis tool n Comparative database and reports n AHRQ Patient Safety Link n E-mail Questions to: Safety. Culture. Surveys@westat. com or Databases. On. Safety. Culture@westat. com Mod 10 2. 0 Page 18 TEAMSTEPPS 05. 2
Measurement AHRQ Patient Safety Indicators (PSIs) n Level IV Results n Provide information on potential in-hospital complications and adverse events n Can help identify potential adverse events that might need further study n Include indicators for in-hospital complications that may represent patient safety events n Quality Indicators Mod 10 2. 0 Page 19 TEAMSTEPPS 05. 2
Measurement Evaluating Team. STEPPS Exercise Three nurses and the infection control officer at a 16 -bed postsurgical intensive care unit attend the 2 -day Team. STEPPS Master Training course. During the implementation planning session, the team agrees that hand hygiene compliance is a major problem, especially physician compliance. As a result, the unit has a relatively high surgical site infection rate. The team decides to use the CUS and Two-Challenge Rule tools as strategies to address this issue. As part of their implementation, they train all the nurses and the employed physician staff on these two tools, but struggle with getting surgeons who are not employed by the hospital to participate. Once the training is complete, they decide to launch the implementation of CUS and the Two-Challenge Rule. Mod 10 2. 0 Page 20 TEAMSTEPPS 05. 2
- Slides: 20