Organizational Readiness to Change Assessment ORCA National QUERI

  • Slides: 15
Download presentation
Organizational Readiness to Change Assessment (ORCA) National QUERI Meeting December 11, 2008 Christian D.

Organizational Readiness to Change Assessment (ORCA) National QUERI Meeting December 11, 2008 Christian D. Helfrich, MPH, Ph. D Implementation Research Coordinator IHD-QUERI

ORCA Organizational readiness to change assessment n 77 -item survey instrument n Fielded among

ORCA Organizational readiness to change assessment n 77 -item survey instrument n Fielded among clinicians and staff involved in planned, discrete clinical change projects n Measures factors that determine effective implementation of a practice change n

ORCA Development Developed by Nancy Sharp and Anne Sales, Ischemic Heart Disease Quality Enhancement

ORCA Development Developed by Nancy Sharp and Anne Sales, Ischemic Heart Disease Quality Enhancement Research Initiative n Based on Promoting Action on Research Implementation in Health Services (PARIHS) framework n

ORCA Evidence Context Facilitation Organizational to change Implementation outcomes

ORCA Evidence Context Facilitation Organizational to change Implementation outcomes

Evidence Scale Discord over evidence Evidence Research Practice experience Context Patient experiences Facilitation Organizational

Evidence Scale Discord over evidence Evidence Research Practice experience Context Patient experiences Facilitation Organizational to change

ORCA Leadership culture Evidence Staff culture Opinion leader culture Context Leadership practice Evaluation /

ORCA Leadership culture Evidence Staff culture Opinion leader culture Context Leadership practice Evaluation / Accountability Slack resources Facilitation Organizational to change

ORCA Leadership roles in planning Project champion roles Evidence Leadership roles in support Implementation

ORCA Leadership roles in planning Project champion roles Evidence Leadership roles in support Implementation team roles Assessment Evaluation Implementation plan Communication Project resources Context Facilitation Organizational to change

Scoring the ORCA Each subscale measured with between 3 – 6 items n Items

Scoring the ORCA Each subscale measured with between 3 – 6 items n Items are statements n Respondents rate level of agreement from 1 (strongly disagree) to 5 (strongly agree) n Subscales are calculated as average of items; scales as average of subscales n Higher scores indicate greater likelihood of success. n

Preliminary validation n Three quality improvement (QI) projects (n = 113): n n n

Preliminary validation n Three quality improvement (QI) projects (n = 113): n n n 1) The Cardiac Care Initiative (n = 65 from 49 facilities); 2) The Lipids Clinical Reminders project (n = 12 from 1 facility) 3) An intensive care unit quality improvement project (n = 36 from 9 facilities) Identical surveys administered to one or more staff involved in quality improvement efforts; Only the initial evidence statement varied, identifying the evidence-based practice change.

ORCA scale reliability Cronbach’s alpha For overall scale For subscales Evidence Context 0. 81

ORCA scale reliability Cronbach’s alpha For overall scale For subscales Evidence Context 0. 81 0. 85 0. 70 – 0. 75 0. 87 – 0. 94 Facilitation 0. 95 0. 84 – 0. 94 Scale

Content validation Exploratory factor analysis n 3 retained factors (eigenvalues 7. 61, 7. 12,

Content validation Exploratory factor analysis n 3 retained factors (eigenvalues 7. 61, 7. 12, 3. 23) n

Principal components analysis Factor 1 Evidence scale Context scale Slack resources -0. 10 –

Principal components analysis Factor 1 Evidence scale Context scale Slack resources -0. 10 – 0. 06 Factor 2 -0. 24 – 0. 11 0. 62 – 0. 83 0. 27 – 0. 67 -0. 17 -0. 07 0. 67 – 0. 88 -0. 08 – 0. 26 0. 18 – 0. 48 0. 41 0. 10 0. 13 0. 71 -0. 10 – 0. 07 0. 13 – 0. 34 Facilitation scale 0. 65 – 0. 92 -0. 14 – 0. 34 Leadership roles in planning Project champion roles Factor 3 Uniqueness 0. 24 0. 74 -0. 02 0. 19 0. 49 0. 35 0. 15 0. 34

Validation Conclusions Overall good reliability and content validity of the three scales n Poor

Validation Conclusions Overall good reliability and content validity of the three scales n Poor reliability of the evidence subscales n Slack resources may be separate factor n Facilitation subscale measuring leadership roles in planning, may be part of context scale; may need rewording n

Next Steps n Submitted grant to conduct further validation n Collaborating with three implementation

Next Steps n Submitted grant to conduct further validation n Collaborating with three implementation projects by Mental Health and Spinal Cord Injury QUERIs to assess predictive validity Develop user’s manual and Web-based, customizable version n Work with partners to produce example cases of how to use organizational readiness assessments to guide implementation projects n

References n Helfrich C, Li Y-F, Sharp ND, Sales A. Organizational readiness to change

References n Helfrich C, Li Y-F, Sharp ND, Sales A. Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARi. HS) framework. Implementation Science. Under review.