Using Evidence to Make Everyday Decisions ROBIN STEED

Using Evidence to Make Everyday Decisions ROBIN STEED, PHD, LOTR LOTA CONFERENCE 2014

Objectives and Agenda Part One • Intro: What? Why? • Learn the 5 easy steps: 1. Identify the problem 2. Ask a question 3. Find some answers 4. Evaluate the answers 5. Try the best one out Part Two �Practice EBP Skills Easy Checklist Getting Started

Part One

Introduction to Evidence Based Practice WHAT DID YOU SAY? NOW WHY WOULD I WANT TO DO THAT?

What is EBP? knowle dge Research Client Factors Clinical Reasoning Best Practice (Sackett , Straus, Richardson, Rosenberg, & Haynes, 2000). wisdom

Why? Client education

Why? Client trust

Why? Do no harm! § NSAIDs do not reduce fibromyalgia pain § No evidence to support psychoanalytic (Freudian) treatment § Stretching does not prevent or improve contracture § Sheltered workshops do not increase employment § Cognitive remediation does not improve function

Why? Your own peace of mind

Why? Our clients deserve the BEST!

The Process of Evidence Based Practice Yes, you too can be an evidenced based practitioner in just 5 easy steps!

Identify the clinical problem 1 I want my clients to: § be more functional § have a higher quality of life § have better outcomes faster § transfer skills to home environment § have maintained gains at follow-up § have better adherence to treatment plan

Types of problems and their questions 1 �Assessment/Diagnostic (What is the best assessment to use to identify performance deficits? ) �Intervention efficacy (What is the best treatment? ) �Intervention cost-effectiveness (Which treatment gives me the most return for my money? )

Identify your problem 1 Complete the problem identification worksheet. Examples: �My clients don’t like constraint induced therapy, would a modified approach work just as well? �My clients have poor sensory motor skills and have trouble with handwriting. �We send instructions home with the clients but we aren’t sure they understand them.

A good problem 1

Ask a PICO question 2 Patient or Problem Intervention Comparison Intervention Outcomes

PICO Examples 2 �In children with autism, evidence on vigorous vs. mild exercise, on stereotyped behaviors? � In clients with schizophrenia, evidence on environmental supports on ability to transition to community? �In children with hemiplegia, evidence on constraint induced therapy on motor function? Work on step two worksheet

Good Questions 2 q Within occupational therapy domain q “What” questions, not ‘why’ q Just right focus, not too wide, not too narrow q Measurable in some way q Clinically important q Not already answered!

Take a break and relax!

Looking for some answers 3 �P: serious or chronic mental illness, psychosis, schizophrenia, mood disorder, anxiety disorder �I: child care, meal preparation, home management, shopping, time management, safety, social participation, education exploration, retirement exploration, employment seeking �O: employment and education (Arbesman & Logsdon 2011)

Types of evidence 3 �Meta-analysis �Systematic review �Critically appraised papers (CAPs) �Critically appraised topics (CATs) �Individual journal articles wikipedia

Where do we find the (free) evidence? 3 AOTA Evidence Exchange Centre for Evidence-based Mental Health Cochrane Consumer Network Mc. Master Occupational Therapy EBR Reviews National Guideline Clearinghouse™ (NGC) OT Seeker - The Occupational Therapy Evidence Database OT Evidence at www. otevidence. info Everyday Evidence Podcast from AOTA Centre for Reviews and Dissemination

Where do we find the articles? 3 �Pub Med ncbi. nlm. nih. gov/pubmed �Pub Med Central: pubmedcentral. gov �Directory of Open Access Journals: www. doaj. org �NBCOT if registered (Pro. Quest) �Google Scholar

Pub. Med Central 3 Open Access: • Arthritis • Autism Research • Journal of Pain Research

NBCOT 3 Pro. Quest Ref. Works �AJOT �An easy way to keep �OTJR �OT International �OT Canada, NZ track of your articles

Google Scholar 3

Critically evaluate the evidence 4 �Read the article �Fill out a form Critical Review from Mc. Master’s University CAP from AOTA CASP in the UK

Critically evaluate the evidence 4 �Study Design randomization control for bias ethical control group �Sample size described �Assessment Properties �Intervention detailed/replicable �Statistical Analysis �Threats to Validity internal external

Make a decision Given the quality of the evidence, what are the implications for clinical practice?

Implement and evaluate 5 Implement Evaluate �What is the risk? �Choose outcome �What is the cost? �Do you need special training? �Getting client consent �Facility support measures �Keep records of outcomes and adverse events �Do a ROI assessment

Take a longer break!

Part Two

Evidence Based Practice 1. Easy Checklist 2. Getting Started

Checklist �Are the results valid? Subject selection Research design Data analysis & results �Are the results meaningful? �Are the results applicable to my client? Source: Portney & Watkins (2009)

How were the subjects selected? 1 �Random sampling �Random assignment �Convenience

Sample Size? 2 not that great <30> much better

Sample Size? 3 Inclusion: who we let in Exclusion: of the ones we let in, who do we kick out

How were participants assigned? 4 Did it result in two equal groups at the start?

Was the design reasonable? 5 • Exercise 2 x week • Constraint Induced Therapy 20 hours/day • A 4 hour workshop on cultural competence

Was the intervention the reason for results? 6 • • • History Measurement Instrument Statistical Regression Maturation Unequal Groups Attrition Multiple Treatments Treatment Diffusion Participant Responses

History

Instrumentation

Statistical Regression

Maturation

Unequal Groups

Mortality/Attrition

Multiple Treatment

Treatment Diffusion

Participant’s Reactions �Artificial environment in effort to control extraneous variables (can’t generalize to real world) �Hawthorne effect- subjects aware of being studied �John Henry effect- subjects get competitive between groups �Novelty effect- increase motivation because tx is new

Was everyone blind to treatment? 7

Did the researchers pick the right assessment? 8 • Toglia Categorization Assessment or the Allen Cognitive Levels to measure change over time. • Using the O’Connor as a measure of functional ability Observation versus self-report Simulation versus natural setting Standardized versus made up test

Were the measures valid and reliable? 9 Validity Reliability �Content �Re-test �Face �Tester �Criterion �Internal Consistency �Concurrent Pearson’s r Alpha Cronbach ICC . 7 and above

How many people dropped out of the study? 10 Why? Percentage http: //www. englishblog. com/2013/10/cartoon-oreos-more-addictive-than-cocaine. html#. Uuqug. LTkro. I

Were the appropriate statistics used? 11 Variabl es Level s Group s n > 30 n < 30 1 2 1 Paired t test Wilcoxon 2 Unpaired t (independent t) Mann-Whitney U 1 One way repeated measures ANOVA Friedman 2 One way ANOVA Kruskal Wallis 1 2 way repeated measures ANOVA 2 + Two way ANOVA 3+ 2 + Source: Portney & Watkins (2009)

Were the results significant? 12 � Source: Portney & Watkins (2009)

Take another break

Are the results meaningful? 13 -16 �Is the intervention feasible, cost effective? �Is there any risk? �Is this within the scope of OT? �What is the ‘take home’ message? www. haven. com www. bioness. com/Healthcare_Professionals/H 200_for_Hand_Paralysis. php

Are the results applicable? 17 -19 �Given the quality of the study, should I apply this to my practice? �Do I need extra training? �Do I have the resources to replicate intervention? �Are subjects similar to my clients?

Get Real! LIKE I AM REALLY GOING TO HAVE TIME FOR ALL THIS!

Why not? Therapists said. . . �Lack of time (87%) �Large workload/caseload (67%) �Limited searching skills (50%) �Limited critical appraisal skills (44%) Mc. Cluskey (2003)

Expert Solutions �Prioritizing daily tasks �Making plans & goals �Delegating tasks �Setting aside blocks of time �Dividing into manageable chunks

Influences on Success �Readiness for change �Personal and organizational expectations �Presence of deadlines �Availability of support

EBP in just 10 minutes a day! �Join AOTA (Journal Club Toolkit) �Listen to AOTA evidence podcast �Read one or two journal articles a month �Look up the quick summaries on EBP web sites �Sign up for evidence alerts at Mc. Master’s �Start an EBP file at work �Join/start a journal club �Keep records

EBP Books on Evidence photos: amazon. com

EBP Books on Implementing EBP

Reasons to Create Evidence �Referrals �Client self-efficacy �Funding for program �Third party payment �Resource allocation �Keep your job !

References � Arbesman, M. , & Logsdon, D. (2011). Occupational therapy interventions for employment and � � � education for adults with serious mental illness: A systematic review. American Journal of Occupational Therapy, 65, 238 -246. Dunn, (2008). Bringing evidence into everyday practice. Thoroughfare, NJ: Slack Mc. Cluskey, A. (2003), Occupational therapists report a low level of knowledge, skill and involvement in evidence-based practice. Australian Occupational Therapy Journal, 50, 3– 12. doi: 10. 1046/j. 1440 -1630. 2003. 00303. x Page, S. J. , Levine, P. , & Hill, V. (2007). Mental practice as a gateway to modified constraint induced movement therapy: A promising combination to improve function. American Journal of Occupational Therapy, 61(3), 321– 327. http: //dx. doi. org/10. 5014/ajot. 61. 3. 321 Portney, L. & Watkins, M. (2009). Foundations of clinical research (3 rd ed. ). Upper Saddle River NJ: Pearson. Sackett, D. L. , Straus, S. E. , Richardson, W. S. , Rosenberg, W. , & Haynes, R. B. (2000). Evidence based medicine: How to practice and teach EBM (2 nd ed. ). London: Churchill Livingstone. Gladstone, D. J. , Danells, C. J. , & Black, S. E. (2002). The Fugl-Meyer Assessment of Motor Recovery after Stroke: A Critical Review of Its Measurement Properties. Neurorehabilitation and Neural Repair, 16(3), 232 -240. doi: 10. 1177/154596802401105171

Resources at LSUHSC Shreveport www. robinsteed. pbworks. com