Using Clinical Data to Answer a Research or

Using Clinical Data to Answer a Research or Quality Improvement Question Family Medicine Residency Network Scholarship Webinar February 2019 Allison Cole, MD, MPH Director, WWAMI region Practice and Research Network Associate Professor, Family Medicine

Learning Objectives • Identify sources of clinical data for research and QI projects • Understanding how to develop research or QI questions that can be answered using clinical data • Understand plan data collection activities to support research and QI efforts

Sources of Data in Clinical Practice • Electronic health records • Patient or provider surveys • Patient, provider or staff interviews 3

Sources of Data in Clinical Practice • Electronic health records • Patient or provider surveys • Patient, provider or staff interviews 4

Card Studies • Brief anonymous questionnaires • Low cost • Observational study • Estimate prevalence of conditions • Get data directly from patients or providers • Assess delivery of care, quality of care, patient or provider preferences Westfall JM et. Al. Card Studies for Observational Research in Practice. Annals of Family Medicine 2011; 9: 63 -68. 5

Card Study Example J Am Board Fam Pract January 1988 1: 15 -23; doi: 10. 3122/jabfm. 1. 1. 15 6

Examples of Card-Study Appropriate Research Questions • What proportion of patients in primary care practice experience food insecurity? • What proportion of patients seen for upper respiratory tract infection symptoms request a prescription for an antibiotic medication? • How often do primary care providers make referrals to complimentary and integrative health services and for what conditions? 7

WPRN Card Study: QUESTIONS 1. What proportion of overweight and obese patients would participate in a comprehensive weight loss program? 2. What are the most important factors in helping overweight and obese patients decide whether or not to participate? 8

Study Methods Patient population: ► Adult (ages 18+) patients ► Pregnant women excluded Implementation ► Administered over 2 week period in June 2013 ► Handed out to patients at front desk check-in Clinics: 12 participating sites across 5 states Survey characteristics: ► Available in English and Spanish ► Anonymous IRB exempt 9

10

11

Surveys Returned By Site Surveys returned Percent of population reached Number returned by site N returned / number of visits 400 361 382 100% 80% 300 205 217 200 131 150 151 60% 170 20% 33 1 8% 5% 0 2 3 4 5 6 Site 7 8 9 10 11 42% 34% 33% 40% 79 92 100 52% 45% 27% 16%16% 0% 1 2 3 4 5 6 7 8 9 10 11 Site 12

Demographics Sex Age 100% Male; 29% 90% 80% 70% 60% 50% 40% 30% 20% 10% 17% 19% 21% 19% 14% 0% 18 -24 25 -34 35 -44 45 -54 55 -64 65+ Female; 71% Missing 0. 3% 13

Overall Health 100% 90% 80% 70% 60% 50% 40% 32% 33% 30% 20% 18% 12% 10% 4% 0% Poor Fair Good Very Good Excellent Missing 1% 14

Insurance Status 100% 90% 80% 70% 60% 50% 40% 30% 27% 20% 14% 10% 18% 10% 16% 10% 2% 0% None Medicaid Medicare Private Other Don't know Multiple Missing 4% 15

Race/ethnicity Pacific Islander 1% Asian 2% Other 3% Black 4% Hispanic or Latino 6% Multiple Races 7% American Indian/Alaska Native 8% White, NOT Hispanic or Latino 67% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Missing 3. 5% 100% 16

Body Mass Index (BMI) 100% 90% 80% 70% 60% 50% 40% 33% 27% 30% 21% 14% 12% 10% 0% Underweight Normal weight Overweight Obese Morbidly obese Missing 3% 17

Patient reported reasons for weight loss What’s the number one reason you want to lose weight? 50% 45% 40% 30% 20% 10% 2% 1% 1% 1% 0% 2% 3% 3% 3% 5% 6% 12% 6% fa jo m in ily tp ro bl em lo s ok be tte r fit ne em ss ot io na fe l el be tte r m ed ic al he al th n pa i y er g s N=1174. 1300+ responses en ot he cl vi ty ng e er lo ot h t fo r co m ag e 0% 18

Would you take part in a weight loss program as described in this survey? 100% 80% 60% 40% 27% 20% 34% 23% 10% 0% Would not take part Maybe would take part Probably would take Very likely would take part N=1243 Missing 2% 19

EHR Data Query Widespread adoption of EHRs in primary care practice EHR-generated data can be helpful for population health management and QI activities Most clinical practices have HIT analysts who can conduct queries ► UW Medicine has created a data repository that can be queried to examine questions relevant to UWNC Northgate and Harborview 20

WHAT EHR DATA CAN AND CAN’T DO Pitfalls to EHR data • Inaccuracy • Incomplete • Badly transformed to make meaningless (e. g. , billing ICD-9 codes) • Unrecoverable for research (e. g. , stored in text only) • Of unknown provenance (i. e. , can’t determine where they came from or what exactly they mean) • Insufficient granularity • Incompatible with research protocols (e. g. , no ability to determine accurate onset or treatment change times frames) Hersh et al. (2014). Recommendations for the Use of Operational Electronic Health Record Data in Comparative Effectiveness Research. e. GEMs (Generating Evidence & Methods to improve patient outcomes): Vol. 1: Iss. 1, Article 14 21

Recommendations for using EHR data in a study: 4 steps to an evidenced based approach 1. Ask an answerable question – Ask whether a test or treatment is efficacious is not enough. We need to know at a minimum the effectiveness of the intervention relative to some alternative approach in a particular patient population or setting. 2. Find the best evidence – The “evidence” / data in EHRs may be incomplete, incorrect, and inconsistent. We need to determine if we have the proper data and, if so, whether they can be applied to answer our question. 3. Critically appraise the evidence – Can we trust the inferences and conclusions from the data? Are there confounding variables in those data? Have the appropriate methods been used? These may be critical with EHR data where assignment of cause and effect could be difficult, if not impossible. The solution likely comes back to asking the right question, i. e. , one for which we can have confidence that we have found the correct answer. 4. Apply it to the patient situation – Can the evidence be applied in our setting? Just as some RCTs answer questions in patient populations very different from those of the clinician making decisions, we must ascertain if the results obtained from this approach can be applied to a specific clinical context. Hersh et al. (2014) 22

Litmus tests for YOUR research questions • Does your question require data from the “whole picture” of care (care occurring outside your practice and therefore EHR)? Don’t ask about completeness of care ideas, given patients may not get all their care in your clinic. • Will your question be affected over time by practice changes that would significantly affect documentation practices? • If your question requires a start and stop period of time, can you reliably determine these times from the data in the EHR? • Have there been significant coding changes over time for the data you’re interested in that may inadvertently look like a clinical change that doesn’t exist? • Are your data available in coded fields or does it have to be mined out of text notes, etc. ? • Do the data provide sufficient granularity that you need for your study? Hersh et al. (2013). Caveats for the Use of Operational Electronic Health Record Data in Comparative Effectiveness Research. Medical Care, 8(3), S 30 -S 37. 23

Research Question: Make it Simple! 1) What proportion of adult patients, seen for a primary care visit in the last year, has been prescribed any opioid medication in the past year? 2) Of adult patients prescribed an opioid medication in the past year, what proportion was prescribed a sedative in the same year? 24

Study Parameters Study period: October 1, 2015 - September 30, 2016 Adult Patients: Ages 18 and over Visit: In-person primary care office visit with a medical provider (MD, DO, NP, PA) Opioid & Sedative Prescriptions: Opioid and sedative medications on lists provided, that had a “prescription” date in study period, regardless of where the prescription originated (e. g. , in clinic or an outside clinic) 25

EHR Data Results Percent of Adult Patients With an Opioid Prescription in One Year Study Period 100, 0% 80, 0% 60, 0% 40, 0% 20, 0% 15, 6% 20, 2% 26, 0% 29, 5% 14, 2% 14, 1% 9, 9% 10, 2% 17, 9% 6, 8% 0, 0% All Sites Site 1 Combined Site 2 Site 3 Site 4 Site 5 Site 6 Site 7 Site 8 Site 9 Site 10 n= 10, 027/ 64, 140 26

EHR Data Results Of Patients with Opioid Prescription: Percent With a Sedative Prescription in One Year Study Period 100% 90% 80% 70% 59, 5% 60% 50% 40% 31, 9% 31, 1% 21, 2% 20% 18, 8% 21, 1% 19, 6% 16, 7% 10, 3% 8, 0% 10% 5, 8% 0% All Sites Combined Site 1 Site 2 Site 3 Site 4 Site 5 Site 6 Site 7 Site 8 Site 9 Site 10 n= 2125/ 10, 027 27

A collaborative group of primary care practices in the WWAMI region committed to research and practice improvement WWAMI region Practice and Research Network University of Washington – Seattle, WA Institute of Translational Health Sciences and Department of Family Medicine

WPRN Mission and Values To improve the health and well-being of patients in their communities, through the conduct of collaborative research that informs and enhances primary care clinical practice. ► Maintain an equal and bi-directional partnership between academic researchers and community practices ► Support practice innovation and quality improvement through research ► Conduct research of importance to member practices 29

WWAMI region Practice and Research Network Over 60 clinics across 28 organizations 18 primary care EHR-connected clinics 30

WPRN Clinical Practices Serve largely rural and/or underserved populations Annual number of outpatient visits from 6, 000 -53, 000/organization (over 770, 000 total) Number of physician providers ranges from 1 to 20 + (over 350 MDs, DOs, NPs, PAs total) Diverse Settings • • 15 Community Health Centers/FQHCs 26 Hospital Affiliated Clinics 17 Rural Health Clinics 2 Other Electronic Health Records • • Epic Centricity Allscripts Others 31

WPRN Coordinating Center Allison Cole, MD, MPH Associate Professor WPRN Director Laura-Mae Baldwin, MD, MPH Professor WPRN Senior Advisor Angelina Kolomiets, MPH ITHS Program Manager WPRN Coordinator Gina Keppel, MPH Research Scientist WPRN Coordinator

Areas of Interest to WPRN members include… ► Use of the electronic health record (EHR) as a tool for research that improves quality of care ► Chronic disease management ► Prevention of adverse drug events ► Cancer screening and prevention ► Chronic pain and opioid addiction ► Behavioral health and primary care integration ► Oral health in primary care ► Mobile health technology

WPRN Champion Activities Meet in person once per year in Seattle, WA Serve as their practice’s point-of-contact with the WPRN Coordinating Center Bring research opportunities to their practice for consideration Share topics of interest to their practice at phone or in person meetings Provide information about their practice to the Coordinating Center on a yearly basis Collaborate on projects that improve practice quality and are published as research and may have the opportunity to be involved in scholarly work resulting from these studies 34

Data QUEST • 18 data-connected clinics in the WPRN An electronic health datasharing architecture across community-based primary care practices in the WPRN • Represents over 150, 000 active patients 35

Thank You

Questions?