BREAKING DOWN EVIDENCEBASED PRACTICES FOR STATE POLICY USING
BREAKING DOWN EVIDENCE-BASED PRACTICES FOR STATE POLICY: USING A COMMON ELEMENTS APPROACH IN PROGRESS NOTE DOCUMENTATION AS AN INDICATOR OF ADHERENCE University of Washington Department of Psychiatry and Behavioral Sciences Sarah Cusworth Walker, Ph. D Georganna Sedlar, Ph. D Jessica Leith, LMFT Lucy Berliner, MSW Cathea Carey, BS Eric Trupin, Ph. D Washington State Department of Behavioral Health and Recovery Paul Davis, MS Felix Rodriguez, Ph. D
The Evidence Based Practice Institute is a intermediary organization that supports EBP translation and implementation efforts with the WA State Department of Social and Health Services Division of Behavioral Health and Recovery (DBHR) for children’s mental health. . Research • Identify gaps in available EBPs based on client need and therapist feedback Policy • Translate EBP for real world contexts • Support CQI feedback loop Training • Broker and develop clinical training to address service gaps
30% State Fiscal Year 2017 benchmark for EBP services in children’s mental health (not an inconsiderable challenge with the national average for EBP use in children’s public mental health at 1 -3%; Bruns, Kerns, Pullmann et al. , 2013)
Defining Evidence Based Practice Source Purpose Level of description Legislative General description References the need for rigorous research design Washington State For inventory review Institute for Public Policy/Evidence Based Practice Institute (EBPI) Specifies the need for costbenefit, heterogeneity in race/ethnic sampling, translational capacity and identifies treatment categories based on metaanalysis Department of Behavioral Health and Recovery (DBHR)/ EBPI Outlines the clinical components in the researchbased treatment categories For provider guidance and monitoring
Monitoring the use of multiple EBPs in a cash-strapped system Billing System • Provides most real time, arguably least burdensome way to document for providers 2013 Self Determination • Allows providers to choose from a number of treatment approaches as long as they are EBP approved 2013 Guidance for Reporting • Removes some of the guesswork for providers in whether they are adherent enough to report accurately 2016 instituted after initial estimates because. . .
Without reporting guidance, numbers were suspect • EBP benchmarks are based on encounters rather than clients • The state error rate for reporting EBPs was 38% with a range across regions of 9 – 83% Error was defined by 1. Reporting non EBP practices for children’s mental health 2. Reporting practices under unrelated encounter types
Designing Reporting Guides Challenge 1: How to establish “good intent” to deliver an EBP Challenge 2: How to allay provider concerns about whether their individualization/modifications of EBPs were eligible for reporting Challenge 3: How to minimize burden and paperwork by keeping all reporting within existing channels (billing and routine progress reporting)
Challenge 1: Establishing “good intent” Training • Provider must have received LIVE training in the reported treatment • Certified trainer or a trainer with documented expertise in treatment approach If reporting a namebrand • Provider can document they are adherent with compliance requirements Treatment Plan • The treatment plan must list intent to provide at least one ESSENTIAL clinical element of the proposed treatment approach
Challenge 2: Allaying provider concerns about fidelity If the state was allowing only a few interventions, questions about adaptation could be fairly well-managed with expert consultation. However, in a complex system with multiple programs, implementation is considered “good enough” for counting if the core philosophy and strategy of the treatment is maintained. This reflects and draws from efforts to identify core components to develop flexible treatment strategies adaptable to real world contexts: T O N , Y L N O S E O P C I R V U that innovation Diebold et al. (2000) P suggest must D A G L A N call "assimmodation, " reflect what they a balance of I C T I R N I O of innovations Lto existing structures and assimilation P C E E R B accommodation of those structures to incorporate key R O O T F D elements. (Elias et al. , 2003) E D N INTE Elias, M. J. , Zins, J. E. , Graczyk, P. A. , & Weissberg, R. P. (2003). Implementation, Sustainability, and Scaling up of Social- Emotional and Academic Innovations in Public Schools. School Psychology Review, 32(3), 303 -319. Kendziora, K. , & Osher, D. (2016). Promoting Childrens’ and Adolescents’ Social and Emotional Development: District Adaptations of a Theory of Action. Journal of Clinical Child and Adolescent Psychology, 45(6), 797 -811. doi: 10. 1080/15374416. 2016. 1197834
Identifying Components • Began with treatment categories as identified by WSIPP in meta- analyses indicated as research-based on the state inventory • e. g. , CBT for Anxious Children • Reviewed available taxonomies, meta-analytic studies and dismantling studies of clinical components for these categories (e. g. below) • • • Chorpita, B. , Daleiden, E. , & Weisz, J. (2005). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7(1), 5 -10. Weisz, J. R. , Chorpita, B. F. , Palinkas, L. A. , Schoenwald, S. K. , Miranda, J. , Bearman, S. K. , . . . Gibbons, R. D. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274 -282. doi: 10. 1001/archgenpsychiatry. 2011. 147 Wright, C. , Catty, J. , Watt, H. , & Burns, T. (2004). A systematic review of home treatment services. The International Journal for Research in Social and Genetic Epidemiology and Mental Health Services, 39(10), 789 -796. doi: 10. 1007/s 00127 -004 -0818 -5 • Consulted with clinical experts to validate and refine the components • “Essential” are 1) designated components that are in at least 80% of effective treatment programs for that category, 2) are reasonably distinct from other treatment category practices, 3) appear to be independently effective. • “Allowable” are clinical components common to effective treatment categories that may or may not cross multiple treatments (e. g. , problem solving).
Challenge 3: Minimize reporting burden and paperwork
Feasibility Evaluation 2017 -2018 Evaluation Plan Adherence: • EBPI will roll out up to 12 regional trainings on using the Reporting Guides. • Participating sites (anticipating 12 -15) will be asked to provide 10 -20 randomly drawn cases subsequent to the training which will be scored for adherence with the RG standard. Acceptability • Following in person trainings, EBPI will send a survey to participants containing the EBP Attitudes Scale and questions about the acceptability of the Reporting Guides in practice. Knowledge: • An instructional web video (link below) will be disseminated through the children’s mental health network with all subcontracted agencies encouraged to have providers view and take a short knowledge-based quiz. • Anticipating 100 -300 responses for assessing knowledge of how to document practice. • All participants in the live training will take the same quiz (anticipated 50 -100), allowing for an assessment of training context on knowledge.
Thank you! For inquiries: Sarah Cusworth Walker secwalkr@uw. edu Acknowledgments: Division of Behavioral Health and Recovery Behavioral Health Organization child care coordinators The Washington State Institute of Public Policy (WSIPP/ Marna Miller) Ron Gengler and staff (Comprehensive) Melissa Gorsuch-Clark and staff (Catholic Family and Child Services) Suzanne Kerns Eric Bruns Greg Endler
- Slides: 15