IMMPACTXXIV HohenschurzSchmidt et al 2020 Oct 22 2020

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Pragmatic Trials of Pain Therapies: a Systematic Review of Methods Conducted for the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) – Presented at IMMPACT-XXIV Virtual Meeting David Hohenschurz-Schmidt, Dr Bethea Kleykamp, Dr Jerry Draper-Rodi, Dr Jan Vollert, Dr Ida Bixho, Jessica Chan, Dr Mc. Kenzie Ferguson, Dr Ewan Mc. Nicol, Jules Phalip, Prof Scott R Evans, Prof Dennis C Turk, Prof Robert H Dworkin, Prof Andrew SC Rice @davidhosch

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Background • Continuum of trial generalizability • Pragmatism in trial design, conduct, and reporting What is the current practice in pragmatic analgesia trials? 57 included trials ü Wide range of interventions ü Comparative effectiveness ü Size variable ü Reporting poor in key areas ü Less pragmatic in certain aspects @davidhosch 2

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Systematic Review Methods • • Pre-registered protocol (CRD 42020178954) Systematic search Population Clinical pain Intervention Comparator Any – Screening & data extraction: in duplicate (excl. within -patient controls) Outcomes Painrelated Study design Prospective RCTs Other Declared pragmatic 2018 -2020 Exclusion criteria • Data extraction – (Pragmatic) Trial methods Feasibility / Pilot trials < 40 participants per arm – Internal validity – Discussion & contextualisation @davidhosch 3

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Systematic Review Methods • Ratings of pragmatism – PRECIS-2 instrument – Good inter-rater reliability and reasonable discriminant validity • Data Synthesis 1 explanatory – Mainly descriptive 3 pragmatic 5 – Correlation analyses – No formal risk of bias assessment – Subgroups The PRagmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) wheel. (Loudon et al. , 2015; doi: 10. 1136/bmj. h 2147) For tool validation see: Loundon et al. , 2017; doi: 10. 1016/j. jclinepi. 2017. 06. 001 @davidhosch 4

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Search Results @davidhosch 5

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Characteristics • Trial objectives • Therapies / interventions • Pain description • Sample size • Trial settings • Funding sources @davidhosch 6

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Characteristics • Trial objectives • Therapies / interventions • Pain description • Sample size • Trial settings • Funding sources 100 most-mentioned words from study objective; Size = n of mentions: largest = 35 mentions; smallest = 2. ; 1750 total words @davidhosch 7

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Characteristics Component therapeutic modalities n 57 total Pharmacological therapy 12 21 % • Trial objectives Cognitive-behavioural & other psychotherapy 9 16 % • Therapies / interventions Surgery 7 12 % • Pain description Acupuncture / acupressure 6 10 % • Sample size Manual Therapy 6 10 % • Physiotherapy 4 7% Trial settings Multidisciplinary care (non-drug) 3 5% • Funding sources General Practice (non-drug) 2 4% Rehabilitation 2 4% Body-mind therapies 2 4% Education 1 2% Automated symptom and treatment side effect monitoring 1 2% Virtual reality 1 2% Dentistry 1 2% @davidhosch 8

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Characteristics • Trial objectives • Therapies / interventions • Pain description • Sample size • Trial settings • Funding sources @davidhosch n Pain disorder / descriptor Back or neck pain Pain duration Peripheral joint pain Acute Arthritis (RA and / or OA) Subacute Pain (not further specified) Chronic Post-medical intervention pain Mixed Abdominal and other visceral pain Not reported Neuropathic pain 57 total 19 57 total 10 12 % 8 4% 7 54 % 5 2% 4 28 % 3 33 % Headaches 3 5% Leg pain 2 4% Post-injury pain 2 4% Tooth pain 1 2% Diffuse chronic pain (CFS, FM, CRPS) 1 2% n 7 2 31 1 16 17 % 14 % 12 % 9% 7% 5% 9

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Characteristics Sample size • Trial objectives • Therapies / interventions • Pain description • Sample size • Trial settings • Funding sources Median Range 234 80 - 1702 This Photo by Unknown Author is licensed under CC BY-SA @davidhosch 10

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Characteristics • Trial objectives • Therapies / interventions • Pain description • Sample size • Trial settings • Funding sources Number of settings n 57 total Single-center 12 21 % Not reported 7 12 % Multi-center (range) 2 - 100 (Median 5) This Photo by Unknown Author is licensed under CC BY-SA @davidhosch 11

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Characteristics • Trial objectives • Therapies / interventions • Pain description • Sample size • Trial settings • Funding sources Funding source n 57 total Industry 7 12 % Public 44 77 % Mixed 5 9% Not reported 1 2% This Photo by Unknown Author is licensed under CC BY-SA @davidhosch 12

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Methods • Comparator group • Blinding • Randomization • Outcome analysis and interpretation @davidhosch 13

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Methods Comparator • Comparator group • Blinding • Randomization • Outcome analysis and interpretation n 57 total Another active specific therapy (CER) 29 51 % Treatment / care as usual (CER) 14 25 % Placebo or sham intervention 5 9% No-treatment group (explicitly assigned) 4 7% Treatment / care as usual plus something else (e. g. advice, education, etc. ) (CER) 2 4% Waitlist control 1 2% Advice only 1 2% Wait & see (not waitlist but monitoring) 1 2% No-treatment group (but unaware of trial) 1 2% CER, Comparative effectiveness research @davidhosch 14

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Methods • Comparator group • Blinding • Randomization • Outcome analysis and interpretation Blinding (as reported) n reported in Participant blinding 13 24 % 54 Provider blinding 4 7% 55 Assessor blinding 45 90 % 50 . @davidhosch 15

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Methods • Comparator group Method of randomization • Blinding • Randomization • Outcome analysis and interpretation @davidhosch n 57 total Individually randomized: 27 47 % Simple randomization 15 26 % Blocked randomization 12 21 % Stratified by site 15 26 % Other stratification 9 16 % Cluster randomized 6 10 % Not reported 0 0% 16

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Trial Methods • Comparator group Trial design and analysis • Blinding • Randomization • Outcome analysis and interpretation n 57 total Superiority trial: 52 91 % ‘Unsuccessful’: 24 Reporting ‘equivalence’ Non-inferiority trial: Employing placebo control group 4 7% 0 Not reported 1 2% Multiple primary outcomes: 15 26 % Not addressing multiplicity @davidhosch 9 9 17

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Pragmatic Trial Methods PRECIS-2 ratings Mean SD 3. 8 0. 6 Overall 1 explanatory 3 pragmatic 5 Mean @davidhosch SD 18

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Pragmatic Trial Methods PRECIS-2 ratings Eligibility Mean SD 4. 0 1. 1 • Low ratings mainly for exclusion of common comorbidities and medications • Low reporting of provider and trial center eligibility Mean @davidhosch SD 19

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Pragmatic Trial Methods PRECIS-2 ratings Recruitment Mean SD 3. 0 1. 6 • Lowest average rating • Recruitment a challenge of (pragmatic) trials • 58% employed targeted recruitment strategies • 27% missed recruitment target Mean @davidhosch SD 20

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Pragmatic Trial Methods PRECIS-2 ratings • Mean SD Setting 3. 8 1. 4 Organisation 3. 5 1. 3 Lack of reporting hampers assessment of generalizability Mean @davidhosch SD 21

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Pragmatic Trial Methods • PRECIS-2 ratings Mean SD Flexibility (delivery) 3. 5 1. 2 Flexibility (adherence) 4. 3 1. 0 Intervention standardized in 61% – Fidelity monitoring in 31% of those • High flexibility in treatment adherence – Post-randomization exclusion only in 9% Mean @davidhosch SD 22

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Pragmatic Trial Methods PRECIS-2 ratings Follow-up Mean SD 3. 2 1. 3 • Often elaborate and more frequent outcome assessments • Average trial duration: 1 year • Average attrition: 15% Mean @davidhosch SD 23

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Pragmatic Trial Methods PRECIS-2 ratings Mean SD Primary outcome 4. 5 1. 0 Primary analysis 4. 3 1. 3 • Mainly patient-reported primary outcomes • 54% also employed objective outcomes • No risk/benefit analysis • Intention-to-treat reported in 84% Mean @davidhosch SD 24

2 5 IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Reporting in Pragmatic Trials • Good reporting on generic CONSORT items reported not reported 100% 75% • Deficits in reporting of generalizability items 50% 25% at io n es ua l Se tti ng co nt ex t ng iz fe at ur ns at io Se tti de co l Pr ot o ce ou r es R vi at te r al w lo io ns if. -u p on te C Fo l & O ut co m e ili ig ib El ju st xt g in fr ria ite cr ty of am ju st ng di bl in ra pa en ce C om A bs @davidhosch if. sc r. to r re & w flo de as on s d. st an n nt io Pa rt i. In te r ve Pr ob l em de f. 0% 25

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Links between Pragmatism and Trial Features • No association between PRECIS-2 ratings and – Funding source – Number of trial centers – Therapy type or pain category • No association between participant blinding and – PRECIS-2 ratings – Funding source – Trial size – Analysis method • Less attrition in larger trials @davidhosch 26

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Links between Pragmatism and Trial Features Subgroup Analyses • Pharmacological vs. non-pharmacological – No difference in overall PRECIS-2 ratings – Less protocol flexibility in drug trials • Acute vs. chronic pain trials – Higher overall pragmatism in acute trials – More recruitment efforts in chronic pain trials @davidhosch 27

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Summary and Discussion Pragmatic trials in pain research… • Exist and are relatively pragmatic • Comply well with general reporting guidelines • Musculoskeletal pain and non-drug interventions dominate • Rarely industry-funded @davidhosch • Challenges partly as in explanatory RCTs (e. g. , recruitment and retention) – May interfere with pragmatic aims • Generalizability can be frustrated by inappropriate reporting 28

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Summary and Discussion • Decide if and where we need more pragmatic trials • Promote core features of pragmatic trial design (PRECIS-2 domains) • Focus more on analysis methods • Improve reporting @davidhosch 29

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Thank you Pragmatic Trials of Pain Therapies: a Systematic Review of Methods David Hohenschurz-Schmidt, Dr Bethea Kleykamp, Dr Jerry Draper-Rodi, Dr Jan Vollert, Dr Ida Bixho, Jessica Chan, Dr Mc. Kenzie Ferguson, Dr Ewan Mc. Nicol, Jules Phalip, Prof Scott R Evans, Prof Dennis C Turk, Prof Robert H Dworkin, Prof Andrew SC Rice @davidhosch 30

IMMPACT-XXIV Hohenschurz-Schmidt et al. , 2020 Oct 22, 2020 Supplementary Slides @davidhosch 31