Resources for EvidenceBased School Social Work Practice Stephanie

Resources for Evidence-Based School Social Work Practice Stephanie Baus Tulane University

Workshop Outline n Overview of Evidence-Based Practice – – n EBP definition and model EBP process (steps) Range of practice questions Resources for learning EBP skills EBP in School Social Work – – Relationship to RTI and PBS Assessment Interventions Your practice questions

Workshop Outline (cont. ) n Formulating practice questions – – – n Effectiveness/prevention Risk/Assessment Descriptive Resources for locating evidence – – – Systematic reviews of studies Electronic databases Search strategies Specialized sites Scholarly books

Workshop Outline (cont. ) n Evidence-informed Assessment in SSW – – n Assessment questions Locating evidence (searching, resources) Critical appraisal Application Evidence-Informed Intervention in SSW – – – Effectiveness/prevention questions Locating evidence (searching, resources) Critical appraisal Application Iatrogenic interventions

Workshop Outline (cont. ) n Evaluating Outcomes – Group designs – Single subject designs

Overview of Evidence-Based Practice

Definition of Evidence-Based Practice “Placing the client’s benefits first, evidencebased practitioners adopt a process of lifelong learning that involves continually posing specific questions of direct practical importance to clients, searching objectively and efficiently for the current best evidence relative to each question, and taking appropriate action guided by evidence” (Gibbs, 2003, p 6).

EBP Model Clinical state and circumstances Practitioner's Expertise Client Preferences and actions Research Evidence Haynes, Devereaux, and Guyatt, 2002

Steps of EBP* n Step 1: Convert an information need into an answerable practice question. n Step 2: Efficiently locate the best evidence to answer the question. n Step 3: Critically appraise the evidence for its validity and usefulness.

Steps of EBP (cont. )* n Step 4: Using practice expertise to integrate evidence with student characteristics and school context, apply the results of the evidence appraisal to practice. n Step 5: Evaluate the outcome of evidence-based action. n Step 6: Teach others: challenges and obstacles *Based on Sackett, et al. , 1997, and Gibbs, 2003

Range of Practice Questions Effectiveness/Prevention n Assessment/Risk n Descriptive n Terminology Evidence-Based Practice (EBP) n Evidence-Informed Practice n Empirically-Supported Interventions/Treatments n

Resources for Learning EBP Skills Gibbs, L. E. (2003). Evidence-based practice for the helping professions. Pacific Grove, CA: Brooks/Cole. Thomson Learning. n Kelly, M. S. , Raines, J. C, Stone, S. & Frey, A. (2010). n n n School social work: An evidence-informed framework for practice. New York: Oxford University Press. Raines, J. C. (2008). Evidence-based practice in school mental health. New York: Oxford university Press. Rubin, A. (2008). Practitioner’s guide to using research for evidence-based practice. Hoboken, NJ: John Wiley & Sons.

Evidence-Based Practice in School Social Work

EBP in Response to Intervention and Positive Behavior Support Models n Efficient and Reliable Assessment Tools/Procedures – – – n Screening to identify prevalence of problems Diagnostic evaluation of individual students Monitoring of treatment fidelity and student progress Empirically-Supported Interventions – Primary Tier: core curriculum to all students/ preventive – Secondary Tier: additional support to at-risk students – Tertiary Tier: intensive support to individual students

Evidence-Informed School Social Work Practice n Recent survey of over 1600 school social workers indicates – Few use online databases, journals, or scholarly books to inform practice. – Primary focus on interventions targeting individual change and risk factors rather than primary prevention. (Kelly, et al. , 2008). n Your practice questions?

Resources for Formulating Practice Questions

COPES Questions Client-Oriented n Practical n Evidence-Search n Gibbs, L. E. (2003). Evidence-based practice for the helping professions: A practical guide with integrated multimedia. Pacific Grove, CA: Brooks/Cole.

COPES Questions Client-Oriented Practical Evidence-Search Four Features of a Well-Built Question Client Type and Problem: What You Might Do: Alternate Course of Action: What You Want to Accomplish: How would I describe a group of clients of similar type? Apply a treatment; act to prevent a problem; survey or interview clients; measure to assess a problem; screen clients to assess risk. What is the main alternative other than in the box to the left, if any? Outcome of treatment or prevention; accurate description of client; valid measure; accurate risk estimation. Gibbs (2003)

Research Question COPES Question Identifies specific sample Describes type of client and problem Identifies particular instance of intervention, including what, where, when, how May describe particular type(s) of interventions or ask about “best intervention” Identifies specific measurement instruments Describes intended outcomes, not measures

Formulating Effectiveness/ Prevention Questions: a) Comparing an intervention to no intervention b) Comparing two interventions c) What is the best intervention? Examples Client/Problem Action Alternate Outcome Rick Ager’s question: If African American adolescents receive Motivational Interviewing or no intervention are the former more likely to decrease their use of alcohol and other drugs? Loretta Pyles’ question: For clients of social service agencies do hierarchical models of agency management or collectively structured management models result in greater client empowerment? Lynn Pearl- For couples who mutter’s have experienced question: one partner’s marital affair what is the best therapeutic approach to rebuild trust?

Formulating Assessment/Risk Questions: a) Comparing an assessment instrument/procedure to no assessment b) Comparing two assessment instruments/procedures c) What is the best assessment instrument/procedure? Examples Client/Problem Action Alternate Outcome Loretta Pyles’ question: Among battered women will the Danger Assessment Scale as opposed to no assessment accurately predict lethality? Judy Lewis’ question In an urban community devastated by a natural disaster does door-todoor contact with residents or outreach to existing institutions in the area a better means of predicting the potential viability of a given community? Eileen Ihrig’s’ question: For children in cross-cultural settings t is the best assessment tool? to assess wha psychosocial wellbeing?

Formulating Descriptive (Quantitative) Questions: a) Summarizing characteristics (how much, how many, what percentage, what is the average? ) b) Asking about relationships between variables Examples Client/Problem Action Cyleste Collins’ question [If] social work students [are surveyed] Rick Ager’s question [If] people who have been affected by a natural disaster [are surveyed] Alternate __________ Outcome what percentage endorse stereotypical attitudes about victims of domestic violence? do they report increased alcohol and other drug use compared to predisaster levels?

Formulating Descriptive (Qualitative) Questions: a) Require narrative responses b) Suggest in-depth explorations, little known phenomena (how would they describe, what is the process, how do they experience? ) Examples Client/Problem Action Loretta Pyles’ question: [If] women who have been battered [are interviewed in depth] Eileen Ihrig’s question: If children in conflict situations who were abducted and became soldiers [are interviewed in depth] Alternate __________ Outcome how do they describe what safety means to them? how do they describe their experiences of reintegrating into the community?

Practice Exercise Describe a practice situation: Ask a practice question: Identify question type: Formulate a COPES question: Client/problem Action Alternative Outcome

Resources for Locating Evidence

Systematic Reviews and Meta-analyses n Campbell Collaboration (Social Work, Education & Criminal Justice http: //www. campbellcollaboration. org/ n National Registry of Evidence-Based Programs and Practices, SAMHSA http: //www. nrepp. samhsa. gov/index. htm n What Works Clearinghouse http: //www. ies. ed. gov/ncee/wwc/

On-line Databases n Subscription databases: Cochrane collaboration, SWAB, SSAB, Psyc. Info, Criminal Justice Ab. , Sociological Ab. , Medline, CINHAIL, etc. n Free access databases: Campbell Collaboration, ERIC, Pubmed, Google Scholar, PILOTS n Government and professional sites: National Center for PTSD at http: //www. ptsd. va. gov/, Information for Practice at http: //ifp. nyu. edu/

Search Planning Worksheet n COPES question n Synonyms n Thesaurus n Search terms n Boolean logic – AND, OR, NOT – “wildcards” (truncation)

Client/Problem Action Alternate Outcome MOLES COPES Question Synonyms Controlled Language/ Thesaurus Search Terms (adapted from Gibbs, 2003, p. 140)

Client/Problem Action COPES Question If adjudicated delinquent juvenile males complete an Outward Bound type of program Alternate Outcome as opposed to no program will the former have lower rates of recidivism? MOLES Synonyms Controlled Language/ Thesaurus Search Terms (adapted from Gibbs, 2003, p. 140)

Client/Problem Action COPES Question If adjudicated delinquent juvenile males Synonyms mandated court ordered youth adolescent complete an Outward Bound type of program wilderness program adventure/ challenge program Alternate Outcome as opposed to no program will the former have lower rates of recidivism? MOLES re-arrest reoffend Controlled Language/ Thesaurus Search Terms (adapted from Gibbs, 2003, p. 140)

Client/Problem Action COPES Question If adjudicated delinquent juvenile males complete an Outward Bound type of program Synonyms mandated wilderness program adventure/ challenge program Controlled delinquency Language/ Thesaurus wilderness therapy outdoor education court ordered youth adolescent Search Terms Alternate Outcome MOLES as opposed will the to no former have program lower rates of recidivism? re-arrest reoffend (adapted from Gibbs, 2003, p. 140)

Client/Problem Action COPES Question If adjudicated delinquent juvenile males complete an Outward Bound type of program Synonyms mandated wilderness program adventure/ challenge program Controlled delinquency Language/ Thesaurus wilderness therapy outdoor education Search Terms “Outward Bound” or wilderness or “adventure program*” or “challenge program*” court ordered youth adolescent (delinquen* or adjudicated or mandated) (juvenile* or youth* or adolescen*) Alternate Outcome MOLES as opposed will the to no former have program lower rates of recidivism? re-arrest reoffend recidivism or reoffend or re-offend or rearrest or re-arrest (adapted from Gibbs, 2003, p. 140)

MOLES Methodology Orienting Locators for Evidence Searches (Gibbs, 2003)

Search Summary #1: meta-anal* or meta anal* or systematic review* or synthesis of studies or study synthesis (6, 807) #2: delinquency prevention #3: #1 AND #2 (250) (5)

meta MOLES AND delinquency prevention 250 5 6807

Effectiveness Questions Prevention Questions Risk/Prognosis Questions Assessment Questions Description Questions (With Qualitative Studies a Subset) Syntheses of Studies Random* OR Controlled Clinical trial* OR Control group* OR Evaluation stud* OR Study design OR Statistical* Significan* OR Double-blind OR Placebo (Random* OR Controlled Clinical trial* OR Control Group* OR Evaluation Stud* OR Study Design OR Statistical* Significan* OR Double Blind OR Placebo) AND Prevent* (Risk Assessment OR Predictive Validity OR Predictive Value OR Receiver Operat* OR ROC OR Sensitivity OR Specificity OR False positive* OR False negative* OR Prognos*) AND Predict* (inter-rater OR Inter-observer OR True positive* OR Specificity OR False Positive* OR False negative* OR Sensitivtiy OR predict* OR Receiver operat* OR ROC) AND (assess* OR diagnos*) (Random* Select* OR Survey OR Representative Sample) meta-anal* OR metaanal* OR Systematic Review* OR synthesis of studies OR Study Synthesis Gibbs, 2003, p. 100 Qualitative Studies Qualitative Study OR Qualitative Analysis OR Content Analysis OR In Depth Interview* OR In. Depth Interview* OR Participant Observation OR Focus Group*

Client/Problem Action COPES Question If adjudicated delinquent juvenile males complete an Outward Bound type of program Synonyms mandated wilderness program adventure/ challenge program Controlled delinquency Language/ Thesaurus wilderness therapy outdoor education Search Terms “Outward Bound” or wilderness or “adventure program*” or “challenge program*” court ordered youth adolescent (delinquen* or adjudicated or mandated) (juvenile* or youth* or adolescen*) Alternate as opposed to no program Outcome MOLES will the former have lower rates of recidivism? Random* or controlled clinical trial* or control group* or evaluation stud* or study design or statistical* significan* or double blind or placebo re-arrest reoffend recidivism or reoffend or re-offend or (adapted rearrest or or from Gibbs, re-arrest “outdoor 2003, p. education” 140)

Client/Problem Action COPES Question If adjudicated delinquent juvenile males complete an Outward Bound type of program Synonyms mandated wilderness program adventure/ challenge program Controlled delinquency Language/ Thesaurus wilderness therapy outdoor education Search Terms “Outward Bound” or wilderness or “adventure program*” or “challenge program*” court ordered youth adolescent (delinquen* or adjudicated or mandated) (juvenile* or youth* or adolescen*) Alternate as opposed to no program Outcome MOLES will the former have lower rates of recidivism? meta-anal* or meta anal or systematic review* or synthesis of studies or study synthesis re-arrest reoffend recidivism or reoffend or re-offend or (adapted rearrest or from Gibbs, or re-arrest 2003, p. “outdoor 140) education”

Descriptive (quantitative) MOLES random* select*” OR “random* sampl*” OR survey OR questionnaire* OR “representative sample” OR “national sample” OR “interview schedule” OR “correlat*” (Baus)

Revising the Search n Keep a search history (databases, terms, hits, relevance) n Too few hits: Broaden search, include all synonyms and combinations, use controlled language, check spelling, check Boolean structure, change databases. (Removing MOLES is last resort. ) n Too many hits: Narrow search, more specific terms, add categories, check Boolean structure. n Hits not relevant: Consider multiple meanings and edit terms, use controlled language, use “not. ”

Evidence-Informed Assessment in School Social Work

Evidence-Informed Assessment n Assessment questions n Resources – Kelley. M. L. , Noell, G. & Reitman, David. (2003). Practitioner’s guide to empirically based measures of school behavior. New York: Kluwer Academic Publishers. – Fischer, J & Corcoran, K. (2007). Measures for clinical practice and research: A sourcebook (4 th Ed. ). New York: Oxford University Press. – www. ets. org/test_link/find_tests/ n Searching

Step 3: Critically Appraising the Evidence

Criteria for Evaluating Effectiveness Studies n Relevance (importance to serving clients, common to practice, feasible, may lead to change) n Evidence quality (control group, randomization, attrition, subjects and raters “blind, ” equivalent groups) n Statistical significance (p <. 05 - unlikely due to chance) n Impact of intervention (effect size, absolute risk reduction, number needed to treat, number needed to harm) (Armstrong, cited in Gibbs, 2003)

Meta-Analyses and Systematic Reviews n Differences between meta-analyses, systematic reviews and narrative reviews n Locating reviews n Critically meta-analyses and systematic appraising meta-analyses and systematic reviews

Criteria for Evaluating Descriptive Studies n Relevance (provides important information about client needs, characteristics, and perceptions) n Evidence quality (random selection or representative sample, sample size, clearly written pretested questions, response rate, appropriate generalization) n Statistical significance for relationships (p <. 05 - unlikely due to chance) n Strength of relationships/effect size

Criteria for Evaluating Assessment/Risk Studies n Relevance (importance to clients, common to practice, feasible, may lead to change) n Ease of use (administration, scoring) n Evidence quality – Reliability: interrater agreement and/or Cronbach’s alpha >. 70 – Validity: comparison with gold standard >. 70, positive predictive value, negative predictive value (Armstrong, cited in Gibbs, 2003)

Criteria for Evaluating Qualitative Studies n Relevance (provides important understanding of client experiences, perspectives, processes) n Evidence quality – appropriate, clearly described methods – theoretical consistency – procedures to enhance trustworthiness* (credibility, dependability, confirmability, transferability) – systematic analysis – no unfounded generalization *Lincoln & Guba, 1985

Use of Checklists/Rating Forms n Rate study quality n Applying reliability n Use explicit criteria increases summary score or not?

Step 4: Applying the Evidence to Practice

Application to Practice n Applicability to client and practice context n No evidence is an answer – role of theory, informed consent n Practice wisdom – integrating art and science, experience, relationship, intuition, and evidence

Application to Specific Clients and Agency Contexts (Slide from Jim Barber, based on Sackett, et al. , 2000) Application Questions Decision Aids & Prompts 1. Is this case so different from those in the literature that you cannot apply their results? Differences can be of: n. Degree (quantitative) and/or n. Type (qualitative) 2. Is the intervention feasible in our setting? Potential obstacles include: n. Agency policy, ethics or mandate n. Cost (including time) n. Agency skill 3. What are the potential benefits and harms to our client? Taking account of cost, ‘outcome valence’ and NNT/H: n. Weigh the seriousness if you don’t 4. What are our client’s values & expectations for both the outcome we are trying to achieve and the intervention we are offering? Assist the client to weigh the intervention’s: n. Acceptability n. Desirability 5. Taking account of the above, does the intervention need to be modified? Modifications can be of: n. Degree (length or intensity) n. Type

Step 5: Teaching Others

Challenges and Obstacles n Misconceptions about EBP n Agency/colleague n Availability n Potential support of databases/articles consequences of knowledge

References Gibbs, L. E. (2003). Evidence-based practice for the helping professions: A practical guide with integrated multimedia. Pacific Grove, CA: Thompson Brooks/Cole. Haynes, R. B. , Devereaux, P. J. & Guyatt, G. H. (2002). Clinical expertise in the era of evidence-based medicine and patient choice. Vox Sanguinis, 83, Suppl 1, 383 -386. Haynes, R. B. , Sackett, D. L. , Gray, J. M. , Cook, D. J. , & Guyatt, G. H. (1996). Transferring evidence from research into practice: 1. The role of clinical care research evidence in clinical decisions. ACP Journal Club, 125 (3), A 14 -A 16. Lincoln, I. & Guba, G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage. Sackett, D. L. , Richardson, W. S. , Rosenberg, W. & Haynes, R. B. (1997). Evidence -based medicine: How to practice and teach EBM. New York: Churchill Livingstone. Sackett, D. L. , Rosenberg, W. , Gray, J. M. , Haynes, R. B. , & Richardson, W. S. , (1996), British Medical Journal, 312, 71 -72. Shlonsky, A. , & Gibbs, L. (2004). Will the real evidence-based practice please stand up? Teaching the process of evidence-based practice to the helping professions. Brief Treatment and Crisis Intervention, 4 (2), 137 -153.

Stephanie Baus sbaus@tulane. edu