Identifying Necessary and Sufficient Conditions for Sustainment of

Identifying Necessary and Sufficient Conditions for Sustainment of Evidence. Based Substance Abuse and Mental Health Programs Sapna J. Mendon, Lawrence A. Palinkas, Suzanne Spear, Juan Villamar, and C. Hendricks Brown

Acknowledgements

Acknowledgements • Center Dir. Frances Harding, CSAP • Center for Substance Abuse Prevention (CSAP) – Division of Community Programs – Division of State Programs • Center for Mental Health Services Administration (CMHS) – Division of Prevention, Traumatic Stress, and Special Programs

Disclaimer • The views, opinions, and content expressed in this presentation do not necessarily reflect the views, opinions, or policies of the Center for Substance Abuse Prevention (CSAP), the Center for Mental Health Services (CMHS), the Substance Abuse Mental Health Services Administration (SAMHSA), or the U. S. Department of Health and Human Services (HHS).

Acknowledgements • Michael Davies • Leanne Giordono

Consoldiated Framework for Implementation Research (CFIR) Damschroder et al. , 2009

How Has the CFIR Been Used? § Int: Knowledge & Beliefs about the program (Gilmer et al. , 2006); Relative advantage (Damschroder and Lowery, 2013) § Outer: patient needs & resources (Gilmer et al. , 2006; Damschroder and Lowery, 2013), external policies & incentives (Gilmer et al. , 2006) § Ind. Characteristics: Personal Attributes (Gilmer et al. , 2006); barriers: clinician’s perception of effectiveness (Green et al. , 2014) § Inner: readiness, org. culture, communication, compatability (Gilmer et al. 2006); barriers: structural/communication issues, clinician availability (Green et al. , 2014); communications, tension for change, relative priority, goals/feedback, learning climate, (Damschroder & Lowery, 2013) § Process: Program champions, integration of services (Green et al. , 2014); Planning for implementation (Damschroder and Lowery, 2013).

Results from parent NIDA R 34 § Top 5 highest rated: § § § Lowest rated: § § needs and resources of the communities being served (97. 4%); program champions (94. 9%); assessment of progress made towards sustainment (94. 7%); access to knowledge and information about the program (92. 3%); knowledge and beliefs about the program (91. 4%) pressures to implement from other states, tribes and communities (21. 1%) and organizational incentives and rewards for implementing program (45. 9%) 15 characteristics in the 51 -75% quartile

Current Study § Research Question: Based on CFIR characteristics, hat are the necessary and sufficient important to sustainment? § Sample § § § 44 key staff member interviews from 4 different SAMHSAfunded mental health and prevention programs: GLS, PPS, SPF-SIG, Stop Act) Missing data Total sample for current study: 39

Methods: Selection of causal conditions 51 -75% quartile: ratings of importance 100 Rate of Importance 90 80 70 60 50 40 30 20 10 0 INT: idea; costs OUT: policies; IN: structure; network value fit CFIR Domains Low High IND: iden; personal PROC: exex; change agents

QCA Process Causal condition must (almost always) be present for outcome to occur Necessity Analysis Sufficiency Analysis Outcome (almost) always occurs when causal condition is present ** causal pathway Ragin, 1987 Data Set Calibration

Methods: Thresholds and Calibration Causal conditions: § § Rating of 0 = no membership Rating of 2 or higher and support from qualitative information = full membership Outcome: § § At least 60% of causal conditions rated as important = full membership (on outcome) Full membership approx. 79%

Truth Table Decision points § Results: § 19 circumstances where some configuration of these conditions were applicable § Frequency of cases with consistency on outcome: > 0. 8 § Of those 19, 12 had a consistency score of 1 (20 cases) § Remaining cases were given a 0 on outcome (based on raw consistency) § All remaining cases with zero membership are deleted

Sufficiency Analysis

Consistency = 1 Coverage =. 45 * INT: idea * PROC: external change agents * IND: personal attributes Progra m Freq. SPFSIG 4 GLS 3 PPS 1 Stop Act 1 * OUT: network Causal (sufficient) Pathway Important for Sustainment IND: identification w/ org IN: fit of values * * IN: structure *

Consistency = 1 Coverage =. 45 * INT: idea * PROC: executing to plan * OUT: network Progra m Freq. GLS 3 PPS 1 SPFSIG 3 Stop Act 1 * INT: costs Causal (sufficient) Pathway Important for Sustainment IND: personal attributes IN: fit of values * * IN: structure *

Consistency = 1 Coverage =. 45 * INT: idea * PROC: executing to plan * IND: personal attributes Progra m Freq. GLS 3 PPS 1 SPFSIG 3 Stop Act 1 * OUT: network Causal (sufficient) Pathway Important for Sustainment IND: identification w/ org IN: fit of values * * IN: structure *

Necessity Analysis: Results

Necessity Analysis (conditions) Condition Threshold (consistency) OUT: Degree of Network > 90% IN: structural characteristics > 90% IN: Fit of values > 90% INT: Where idea came from 80 -89% IND: Other personal attributes 80 -89% PROC: External change agents 80 -89% PROC: Executing program according to plan 80 -89%

Discussion and Implications § Network and potential mechanisms of network § Age and maturity are important to sustainment, but…. §Alignment of values– of program to individuals, to existing workflows; presumably stronger alignment § Intervention components and process are necessary- How is information (int. ) and approach to implementation being discussed? § Personal attributes: are there ways leadership can promote motivation, learning, ability to tolerate ambiguity?

Next Steps § Run as fuzzy set analysis (accounting for partial membership) § Develop complex causal pathways (from sufficiency analysis) §Identify necessary pathways

THANK YOU! smendon@usc. edu