Systematic multimethod assessment of adaptations in five health

  • Slides: 23
Download presentation
Systematic multi-method assessment of adaptations in five health system interventions 4 th Biennial Conference

Systematic multi-method assessment of adaptations in five health system interventions 4 th Biennial Conference for the Society for Implementation Research Collaboration Seattle, September 7 -9, 2017 Borsika A. Rabin Marina Mc. Creight Russell Glasgow

Acknowledgment • Implementation Team - Triple Aim QUERI (TAQ), Denver VA (PI: Catherine Battaglia)

Acknowledgment • Implementation Team - Triple Aim QUERI (TAQ), Denver VA (PI: Catherine Battaglia) – VA QUERI funding for TAQ • Transition Nurse Program Team, Denver VA (PI: Robert Burke) – funding from VA Office of Rural Health, N 19 FY 14 Q 3 -S 0 -P 01240 • Cindy Banh and Timothy Cutler from University of California Davis (PCMH Pharmacist Project) – funding from UCSF Divine Family Endowment in Clinical Pharmacy • Special thanks to Charlotte Nolan for drafting and to Charlotte Nolan and Roman Ayele for piloting the ‘real-time’ tracking system

Overview • Defining adaptations and addressing the balance between fidelity and adaptation • Conceptualizing

Overview • Defining adaptations and addressing the balance between fidelity and adaptation • Conceptualizing and assessing adaptations • Adaptations in five health system interventions • Lessons learned and next steps

Adaptation defined #1: Adaptation can be defined as the deliberate or accidental (i. e.

Adaptation defined #1: Adaptation can be defined as the deliberate or accidental (i. e. , drift) modification of the program #2: Adaptation of programs often occur to improve the fit (or compatibility) of a program to a new setting or to increase the cultural appropriateness of a program. #3: Adaptations to evidence-based interventions are common and (some researchers suggest) inevitable to meet the needs of a specific context. #4: Adaptations might lessen the effectiveness of the program if they compromise the core elements and underlying logic of a program. 1 http: //www. csun. edu/sites/default/files/Finding. Balance 1. pdf 2 Carvalho et al. J Public Health Manag Pract 2013; 19(4): 348 -56.

Historical view of fidelity and adaptation

Historical view of fidelity and adaptation

A mature view of fidelity and adaptation Attention to BOTH program fidelity and adaptation

A mature view of fidelity and adaptation Attention to BOTH program fidelity and adaptation during the complex process of program implementation is critical to successful, sustained implementation of evidencebased programs. .

Balancing fidelity and adaptation #1: Identify core components and flexible components of the intervention:

Balancing fidelity and adaptation #1: Identify core components and flexible components of the intervention: theories and frameworks and core component analysis #2: Choose designs that address both internal and external validity #3: Engage stakeholders early and throughout #4: Make adaptations intentional rather than accidental through planning #5: Assess and document fidelity and adaptation throughout the process

Goals of documenting adaptations • Provide contextual process data to interpret outcomes (i. e.

Goals of documenting adaptations • Provide contextual process data to interpret outcomes (i. e. , how adaptations contribute to outcomes) • Consider refinements to the recommended intervention & implementation strategies based on observed changes • Create an organized list of adaptations that future implementers can consider for success • To develop and test replicable, easy-to-use documentation methods for recording adaptations

Adapting the Stirman framework using the RE-AIM model and clinical experience WHY – What

Adapting the Stirman framework using the RE-AIM model and clinical experience WHY – What is the purpose of the adaptation? IMPACT – What are (subjective) short term results of the • Increase reach, participation, access adaptation? • Increase effectiveness • Are they positive, negative, no real impact? • Increase adoption by more clinics/settings or • Did the changes impact: make intervention more aligned with organizational • Reach/participation/access goals • Effectiveness • Increase implementation/ability of staff to deliver • Adoption intervention successfully • Implementation/ability of staff to deliver intervention • Increase maintenance – to make intervention successfully more SW likely institutionalized • Maintenance Stirman etto al. be Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science 2013, 8: 65. Hall et al. Understanding adaptations to patient-centered medical home activities: The PCMH adaptations model Translational Behavioral Medicine, 2017, DOI /10. 1007 s 13142 -017 -0511 -3

Methods to assess adaptation #1: Observational techniques #2: Focused interviews #3: Questionnaires, checklists, and

Methods to assess adaptation #1: Observational techniques #2: Focused interviews #3: Questionnaires, checklists, and logs #4: Content analysis of key documents and curricula #5: Study databases and clinical databases

Sample interview questions WHAT component or part of the intervention was changed in this

Sample interview questions WHAT component or part of the intervention was changed in this adaptation; in other words, what was the nature of the change? (For instance, was it a change to program content, format, delivery mode, staff delivering it, patients eligible, where, when or how it was delivered, or what? ) WHO was responsible for first suggesting or initiating this change? (Was this the person or persons the ones who implemented the change? (If not, who implemented the adaptation? )) WHEN during the ____ program was this adaptation first made? WHY was this adaptation made? (For example, to get more people to participate, to make the program attractive to more settings, to increase its effectiveness, to make it easier to deliver, to make it easier to maintain or reduce costs, etc. ? )

Exampleofof. Tracking trackingform Example

Exampleofof. Tracking trackingform Example

5 health systems interventions Study name Primary Care Network Pharmacist Program Adaptation assessment Timing

5 health systems interventions Study name Primary Care Network Pharmacist Program Adaptation assessment Timing Method Retrospective Interviews Patient Reported Health Status Prospective Interviews Tracking Multi-modal Pain Care Prospective Interviews Tracking Transition of Care Prospective Interviews Tracking Transitions Nurse Program Prospective Interviews Tracking

5 health systems interventions Study name Primary Care Network Pharmacist Program Adaptation assessment Timing

5 health systems interventions Study name Primary Care Network Pharmacist Program Adaptation assessment Timing Method Retrospective Interviews Patient Reported Health Status Prospective Interviews Tracking Multi-modal Pain Care Prospective Interviews Tracking Transition of Care Prospective Interviews Tracking Transitions Nurse Program Prospective Interviews Tracking

Primary Care Network Pharmacist Program • Multiple Primary Care Network clinics affiliated with UC

Primary Care Network Pharmacist Program • Multiple Primary Care Network clinics affiliated with UC Davies • Identify and describe barriers and facilitators of, and adaptations and modifications made during the implementation of a clinical pharmacistbased program • Semi-structured interviews with multiple stakeholders (n=14) • Retrospective assessment of adaptations • Major adaptation (sample): • Chance in number of clinics and pharmacists • Change in program management • Adaptations were used to: • Improve local program implementation and delivery • Inform future scale out within and outside of UC Davis

5 health systems interventions Study name Primary Care Network Pharmacist Program Adaptation assessment Timing

5 health systems interventions Study name Primary Care Network Pharmacist Program Adaptation assessment Timing Method Retrospective Interviews Patient Reported Health Status Prospective Interviews Tracking Multi-modal Pain Care Prospective Interviews Tracking Transition of Care Prospective Interviews Tracking Transitions Nurse Program Prospective Interviews Tracking

Examples of documented adaptations • Change in who contacts patient (task shifting) • Change

Examples of documented adaptations • Change in who contacts patient (task shifting) • Change in recruitment materials and protocol (change in intervention) • Change in feedback procedures (implementation strategy change) • QUERI

Lessons learned: Strengths • • • Adaptation data can lead to actionable information for

Lessons learned: Strengths • • • Adaptation data can lead to actionable information for program improvement and scale out Rapid and frequent assessment of modifications for QI that can quickly inform other assessment methods Does not require clinical staff time (or minimal) Flexible: can be based on observations, team meetings, conversations or other inputs Can modify the coding categories or options based on results (depending on whether QI or research) to fit your project General system has worked across diverse projects

Lessons learned: Limitations • Requires in-depth knowledge of intervention and implementation strategies • (In

Lessons learned: Limitations • Requires in-depth knowledge of intervention and implementation strategies • (In our case) initial differentiation between intervention and implementation strategy unclear • Coding reliability not assessed thus far • Response categories need adjusting so ‘moving assessment’ and difficult to do some types of comparisons

Next steps Rapid methods to assess adaptation so it can inform current project Continue

Next steps Rapid methods to assess adaptation so it can inform current project Continue reflecting on best methods to collect data on adaptations (i. e. , approach, timing, frequency, participants, etc. ) Methods to integrate data from multiple sources Start to separate intentional and unintentional adaptations as well as adaptations to the intervention and implementation strategy Understanding patterns of adaptations across settings/program types through routine documentation/reporting of adaptations

QUERI-wide Adaptation, Fidelity, and Tailoring Special Interest Group • Started in December 2015 and

QUERI-wide Adaptation, Fidelity, and Tailoring Special Interest Group • Started in December 2015 and meets monthly, co-led by Drs. Borsika Rabin and Russ Glasgow • 30+ members from a number of QUERI Programs and beyond (open membership – contact Borsika if interested in joining) • Focuses on best practices on documenting and reporting fidelity, adaptations and modifications for both intervention and implementation fidelity and identification of existing and developing new pragmatic measurement instruments for these areas. • Serves as a Work In Progress forum for colleagues to share and discuss ideas.

“Implementing a program is like constructing a building. An architect draws upon general engineering

“Implementing a program is like constructing a building. An architect draws upon general engineering principles (theory) to design a building that will serve the purposes for which it is designed. However, the specific building that results is strongly influenced by parameters of the building site, such as the lot size, the nature of the site’s geological features, the composition of the soil, the incline of the surface, the stability and extremes of climate, zoning regulations, and cost of labor and materials. The architect must combine architectural principles with site parameters to design a specific building for a specific purpose on a specific site. . This dynamic is mirrored in the rough-and-tumble world of the human services. Despite excellent plans and experience, ongoing redesign and adjustment may be necessary. ” Bauman et al. 1991

WHAT DO YOU THINK? GET IN TOUCH: borsika. a. rabin@gmail. com

WHAT DO YOU THINK? GET IN TOUCH: borsika. a. rabin@gmail. com