KNOWLEDGE TRANSLATION CANADA Making Guidelines Easier to Follow

KNOWLEDGE TRANSLATION CANADA Making Guidelines Easier to Follow: Bridging Best Evidence and a Clear Message Onil Bhattacharyya, MD, Ph. D and Monika Kastner, Ph. D Li Ka Shing Knowledge Institute of St. Michael’s Hospital University of Toronto March 11, 2010

Collaborators Elizabeth Estey n Laure Perrier n Sharon Straus n Merrick Zwarenstein n Jeremy Grimshaw n Ian Graham n Canadian Diabetes Association (CDA) n Canadian Thoracic Society (CTS) n C-CHANGE Funded by KT Canada Clinical Research Initiative (CFI/CIHR) n

Presentation Outline n n Project objectives Background Current guideline tools Study framework and design

Background “Guidelines are often too narrowly focused on single diseases and are not patient focused. Patients seldom have single diseases and few if any guidelines help physicians manage complexity. Paradoxically, guidelines are often too comprehensive, covering every possible intervention that could be possible for a patient with that single disease…. If there is a main message in such guidelines, it is likely to be lost in the minutiae” 1 1 Shaneyfelt TM, et al. JAMA 2009; 301(8): 868 -869.

Background- Guidelines on the frontlines “I’m stumbling because I haven’t gotten to the point where I’ve assigned greater or lesser importance to one or the other, like there’s a number of them and I try to do them all, but if I had to choose one, I’m not sure what to choose, maybe an aspirin (laugh)… I don’t know what criteria I would use to, right now, to prioritize them. ” 2 2 SLZ study participant (MD 1).

Guideline Implementation n Extrinsic strategies ¡ ¡ n Changing environment/providers Highly variable costs and impact Intrinsic strategies ¡ ¡ ¡ Changing guidelines themselves Implementability: a set of perceived characteristics of guidelines that predict the relative ease of their implementation in practice 3 Potentially highly cost-effective 3 Shiffman RN, et al. BMC Med Inform Decis Mak 2005; 5: 23.

Assessment of Existing Tools Existing guideline tools can… n n n Assess the methodological quality of guidelines (AGREE)4 Inform developers about implementation issues (GLIA)5 Adapting existing guidelines to other settings (ADAPTE)6 But do not completely… n n n Consider how content, format and structure could be modified to increase uptake Operationalize the processes that could enhance guideline implementability Identify and resolve the differing views of guideline developers and end users (i. e. providers) 4 AGREE 5 Shiffman Collaboration; Qual Saf Health Care 2003; 12: 18 -23 RN, et al. BMC Med Inform Decis Mak 2005: 5: 23; 6 Fervers B, et al. In J Qual Health Care 2006; 18: 167 -76.

Project Objectives n To determine whether changing the attributes of clinical practice guidelines (CPGs) will improve implementation ¡ ¡ ¡ Do literature synthesis on guideline characteristics and uptake Incorporate perceptions of guideline developers and end users Develop and evaluate a new guideline implementability tool (GUIDE-IT)

Project Objectives n To determine whether changing the attributes of clinical practice guidelines (CPGs) will improve implementation ¡ ¡ ¡ Do literature synthesis on guideline characteristics and uptake Incorporate perceptions of guideline developers and end users Develop and evaluate a new guideline implementability tool (GUIDE-IT)

Guiding Frameworks 1. Graham’s “Knowledge-to. Action” (KTA) framework 2. Medical Research Council (MRC) framework for complex interventions

Study Design Overview 1: Literature review (KTA Steps 1 -2; MRC Phase 1) Problem: Guidelines hard to implement 2: Tool development process (KTA steps 3 -4; MRC Phase 2) Implementability knowledge synthesis Design conceptual tool Test in focus groups Use tool to revise guideline Build prototype 3: Process Evaluation (KTA steps 5 -7; MRC Phase 3) Monitor use of tool Test usability Evaluate impact in RCT Sustain use of tool

Literature review 1: Literature review (KTA Steps 1 -2; MRC Phase 1) Problem: Guidelines hard to implement 2: Tool development process (KTA steps 3 -4; MRC Phase 2) Implementability knowledge synthesis Design conceptual tool Test in focus groups Use tool to revise guideline Build prototype 3: Process Evaluation (KTA steps 5 -7; MRC Phase 3) Monitor use of tool Test usability Evaluate impact in RCT Sustain use of tool

Evidence synthesis n n n Traditional systematic review may miss literature which is multidisciplinary, multifaceted, and under-theorized Synthesis draws on different approaches to combine qualitative and quantitative literature Core question: ¡ What characteristics of guidelines affect uptake?

Approaches to Literature Synthesis Searching Idealist Realist Iterative Linear Quality Content vs. Clear & a assessmen method priori t Problemati Yes No zing Barnett-Page and Thomas, BMC Med Res literature Method, 2009

Approaches to Literature Synthesis Idealist Realist Question Explore Answer Heterogen eity Lots Little Synthetic product Complex Clear, actionable Barnett-Page and Thomas, BMC Med Res Method, 2009

Realist Review n n Explicitly theory-driven approach to evidence synthesis Underlying theory = modifying characteristics of guidelines will increase their uptake Develop a theoretical framework for implementability Pawson RT, et al. J Health Serv Res Policy 2005; 10: 21 -34

Meta-narrative Review n n n Unfolding “storylines” of research in a particular scientific tradition Body of theoretical knowledge and linked set of primary studies Analysis across different disciplines Develop a narrative account of implementability Greenhalgh, T, et al. Soc Sci & Med 2005; 61: 417 -30

Search Strategy 1. 2. 3. 4. Preliminary set of core articles “Snowball sampling” Related-articles search Electronic database search

Preliminary Search & Data Collection Type of Strategy Yield Comments Core Articles 27 Data collection in process Expert consultation 31 Consultation with 4 experts to date Reference tracking 135 Scanning of 27 core articles and 29 expert articles ~1500 Using Pub. Med Related Articles feature Traditional Database Search 11 Developing inclusion/exclusion criteria and collecting search terms from core articles to refine search TOTAL ~1704 Snowball sampling Related Articles

Preliminary Findings n Series of guideline attributes drawn from core articles: ¡ Actionable, clear, specific, complex, evidence-based, flexible, feasible ¡ Conceptualized as a series of trade-offs ¡ Attributes of recommendations vs. guideline as a whole

Preliminary Framework • Actionable TRADE-OFFS • Complex • Clear • Flexible • Specific • Evidence-based FEASIBILITY Guideline Uptake

Analysis n Goal: Develop a conceptual model of implementability ¡ n n Identify and define key dimensions Integrative vs. interpretive reviews Draw on multidisciplinary perspectives ¡ Map theories and narratives within and across each discipline

Multidisciplinary Perspectives Psychology: Cognitive features of the user Gist, Fuzzy Trace Theory Diffusion of Innovations: Characteristics of the “thing” (ie. CPGs, the innovation) ? Human Factors Engineering: Human systems integration Management/Marketing: Persuasive communication

Integrative analysis Diffusion of innovations Communication/ Medicine Psychology • Complexity • Clarity • Evidence-based Fuzzy Trace Theory • Promote gist vs. verbatim memory Persuasive Communication Management/ Marketing Human Factors Engineering • Informative vs. persuasive communication • Directive recommendations Human systems integration • Learnability • Self-confidence • Usability Major Implementability Attributes • Actionable • Clear • Feasible • Complex • Evidence-based

Tool Development Process 1: Literature review 2: Tool development process KTA Problem: KTA Knowledge: Guidelines hard to implement Implementability knowledge synthesis Design conceptual tool Test in focus groups Build prototype Assess barriers to using the tool Test usability

Conceptual tool design n Conceptual design of a guideline implementability tool (GUIDE-IT) ¡ Key guideline attributes from review ¡ Implementability elements of existing tools ¡ Input from content experts

Focus Groups n Focus groups with guideline developers and end-users to assess ¡ ¡ ¡ Perception of guideline development process What is important in guidelines What guideline attributes facilitate/hinder uptake

FG Study Flow Focus Groups Focus Group 1 Focus Group 2 Focus Group 3 Focus Group 4 • Guideline developers • Revise draft guideline • Guideline users (providers) • Comment on revised guideline • Mixed group • Resolve conflicts about implementability dimensions • Mixed group • Feedback on tool design

Prototyping GUIDE-IT n Transform the conceptual design of GUIDEIT into a functioning prototype using: ¡ ¡ ¡ FG findings Implementability components of existing tools Consultation with content experts

Usability testing 1. Providers work through standardized problems with draft guideline 2. Provider feedback to guideline developers, who use tool to revise recommendations 3. Tool tested with both groups to identify standard usability problems Kushniruk AW & Patel VL. J Biomed Inform 2004; 37(1): 56 -76.

Process Evaluation 1: Literature review Problem: Guidelines hard to implement 2: Tool development process Implementability knowledge synthesis Design conceptual tool Test in focus groups Use tool to revise guideline Build prototype Assess barriers to using the tool 3: Tool testing Monitor use of tool Test usability Evaluate impact in RCT Sustain use of tool

Implement, Monitor, Evaluate n Objectives: ¡ ¡ To test the use of GUIDE-IT To monitor its use through a controlled trial

Evaluation Framework n Old and new versions of guideline recommendations (modified using GUIDEIT) presented on a web-based platform n Study comparing impact of original or revised recommendations on clinical decision making n RCT study design based on previous work by Shekelle et al 10 10 Shekelle PG, et al. Health Services Research 2000; 34(7): 1429 -48.

Proposed RCT Study Design Excluded N=? Assessed for eligibility N=? Randomized N=? Group A Vignettes + Standard recommendations N=? Group B Vignettes + Revised recommendations N=? Group C (Control) Vignettes only (no recommendations) N=? Follow-up N=? Analysis N=?

Areas for discussion 1. Realist review & Meta-narrative review: ¡ What do you think of the structure of the review? ¡ Response to initial framework 2. Assessment of guideline tools: ¡ Are we missing any existing tools?

Areas for Discussion 3. Tool design: ¡ How do you envision GUIDE-IT? ¡ What do you think it should look like? 4. Overall study design ¡ What are your thoughts on the methods and approach?