SteppedWedge Cluster Randomised Trials Dr Patty Chondros Department

Stepped-Wedge Cluster Randomised Trials Dr Patty Chondros Department of General Practice The University of Melbourne ACTA Registry Randomised Trials, 19 May 2020 ACTA gratefully acknowledges operational funding from the Australian Government’s Medical Research Future Fund

Outline • Registry data in primary care research • Cluster randomised trials (CRT) • Stepped Wedge design (SW-CRT) • Example: The CORE Study • Design considerations • Summary www. clinicaltrialsalliance. org. au

Registry data in primary care research • Large general practice datasets 1 are held – NPS Medicine. Insight (https: //www. nps. org. au/medicine-insight/using-medicineinsight-data) – Primary Health Networks – Departments of General Practice (for example, the UNSW electronic Practice Based Research Network and the Data for Decisions program at the University of Melbourne) • Through data linkage with hospital data and registries can follow the patient journeys • National Primary Health Care Data Asset under development, AIHW (https: //consultation. aihw. gov. au/phcdu/national-primary-health-care-data-asset/) Manski-Nankervis JE, Sturgiss EA, Liaw ST, Spurling GK, Mazza D: General Practice research: www. clinicaltrialsalliance. org. au an investment to improve health of all Australians, MJA 2020

Data linkage with GP EMR and other registries Aim: Understand the management of osteoarthritis in general practice NPS GP EMR Data (N=475, 870 at 2018) Individuals who attended general practice in 2013 AOANJRR All Total Knee Reconstructions in Australia since 2000 Bio. Grid Australia National Death Index All death since 1996 Linked data set GP patients underwent TKR in study period AOANJRR - Australian Orthopaedic Association National Joint Replacement Registry TKR – Total knee reconstruction www. clinicaltrialsalliance. org. au

Cluster randomised trials (CRT) • CRTs randomise clusters of individuals to intervention or control arms Example: general practice, school, community • Outcomes measured at the individual level • Common in primary care research – intervention is targeted at the general practice (GP) – avoid contamination Example: Implementing guidelines, GP training/system change www. clinicaltrialsalliance. org. au

What is a Stepped-Wedge CRT? • Special type of CRT with a cross-over design in one direction, also described as – – Phased or staggered implementation Multiple baseline Dynamic wait-listed Wait list designs • Increasingly been used to 1, 2 – – – evaluate health policy and service research data driven quality improvement implementation studies pragmatic / effectiveness trials (does it work in practice) complex health care evaluations 1. Grayling MJ, Wason JM, Mander AP Grayling MJ, Wason JM, Mander reporting quality and design features. Trials. 2017 2. Taljaard M, Hemming K, Shah L, et al. Inadequacy of ethical conduct and reporting of stepped wedge cluster randomized trials: Results from a www. clinicaltrialsalliance. org. au systematic review. Clinical Trials 2017

Registry data in primary care research Aim: Evaluate a data-driven quality (DPIQ) improvement intervention in primary care to reduce a composite of nine previously validated measures of high-risk prescribing 40 volunteer general practices in two Scottish health boards Medication use automatically extracted from GP electronic medical records www. clinicaltrialsalliance. org. au

Stepped Wedge CRT (SW-CRT) • All clusters start in the control phase • Randomisation determines the order in which the intervention is implemented • Clusters (or groups of clusters) are randomly assigned to a time-point when they cross-over from control to intervention phase (step/sequence/arm) • At end of the trial all clusters are in the intervention phase • Clusters followed-up longitudinally • Outcomes/endpoints usually measured at discrete time points on individuals www. clinicaltrialsalliance. org. au

Randomised group (Arm) Schematic of a SW-CRT www. clinicaltrialsalliance. org. au Replicated from (page 2) Barker D, Mc. Elduff P, D’Este C, Campbell MJ. Stepped wedge cluster randomised trials: are view of the statistical methodology used and available. BMC Research Methodology 2016

Example: The CORE study 1 Aim: To test the effectiveness of an Experience Based Co-design intervention to improve psychosocial recovery of people with serious mental illnesses who attend Mental Health Community Support Service (MCHSS) Population: People with serious mental illnesses who attend MCHSS (cluster) in Victoria Intervention: Experience Based Co-design intervention involved staff, service users and carers participating in a facilitated process to collaboratively co-design improvements for better service experiences with the goal of optimising recovery-orientation of services. Comparator: Usual care Primary outcome: Psychosocial recovery (24 -item Revised Recovery Assessment Scale, range of score 24 to 120)2 1. Palmer V, Chondros P, Piper D et al. The CORE study protocol: a stepped wedge cluster randomised controlled trial to test a co-design technique to optimise psychosocial recovery outcomes for people affected by mental illness in the community mental health setting. BMJ Open 2015 2. Corrigan PW, Salzer M, Ralph RO, Sangster Y, Keck L. Examining the factor structure of the recovery assessment scale. Schizophr Bull. www. clinicaltrialsalliance. org. au 2004; 30(4): 1035 -1041

The CORE study: Rationale for the SW design • Limited number of MCHSS available – parallel CRT was not feasible • Complex intervention at organisational/service level • Intervention intensive – could not be delivered concurrently to all clusters • Vulnerable population – considered that it was not ethical to withhold the intervention perceived to be beneficial • Stepped-wedge design was preferable to pre-post design • Funders preference www. clinicaltrialsalliance. org. au

Example: The CORE study Follow-up time (period) Arm T 0 T 1 T 2 T 3 3 Control 9 months 2 Control 9 months 18 months 1 Control 9 months 18 months 27 months Three clusters randomised to each arm Arm is the allocation of group of clusters/individuals (four time points/periods) Length of step = 9 months, approximately 4 years to complete 9, 18, 27 months=Intervention phases (numbers indicate length of time since start of the intervention) www. clinicaltrialsalliance. org. au

Reproduced from: Palmer V, Chondros P, Piper D et al. The CORE study protocol: a stepped wedge cluster randomised controlled trial to test a cowww. clinicaltrialsalliance. org. au design technique to optimise psychosocial recovery outcomes for people affected by mental illness in the community mental health setting. BMJ Open 2015

Registrybased data used for the health economics analysis Reproduced from: Palmer V, Chondros P, Piper D et al. The CORE study protocol: a stepped wedge cluster randomised controlled trial to test a cowww. clinicaltrialsalliance. org. au design technique to optimise psychosocial recovery outcomes for people affected by mental illness in the community mental health setting. BMJ Open 2015

Design features of the stepped-wedge CRT Cross-sectional data – Independent samples at each time-point – Different individuals are measured at each time point Closed cohort (longitudinal) data – Repeated measures on the same individual at each time point, no new individuals can join after the study starts Open cohort – Individuals have repeated measures but new individuals can join over the study duration www. clinicaltrialsalliance. org. au

CORE Study: Repeated cross-sectional or cohort samples? Repeated cross-sectional samples not practical • Extremely difficult to reach population • Small cluster size (60 to 350) – higher chance of sampling the same individuals over time • Increased chance of selection bias – If recruiters could not be blinded to the intervention allocation in later stages • Recruitment strategies were costly and time consuming – Research assistant face to face recruitment at the MHCSS and staff raising awareness about the study www. clinicaltrialsalliance. org. au

CORE study: Closed or open cohort design? • Client lists vary from 60 -350 per MHCSS (cluster), but not all clients will be engaged in a program at any one time • Over the length of the study, clients may leave the MHCSS & new clients may register Thus, CORE study employed an open cohort • Individuals were sampled from each MHCSS prior to randomisation and followed at each time point • New individuals were recruited at the beginning of each step, which also offset attrition www. clinicaltrialsalliance. org. au

CORE study: Open cohort design • Selection bias minimised as participants recruited prior to randomisation • Each step was 9 months, some clusters waited a long time before starting intervention • Motivation to participate in study may decrease • Attrition rate over time • Participants do not attend the MCHSS (cluster) after the intervention was implemented (not exposed to the intervention) • Intervention effect attenuated • Recruitment of participants may take longer than anticipated Example: 9 months instead of 3 -6 months www. clinicaltrialsalliance. org. au

Design considerations for the SW-CRT Study design • Number of clusters • Number of individuals per cluster • Number and length of the steps • Number of clusters randomised at each step • Cross-sectional or cohort design (closed/open) Logistical / Practical • Number of suitable/available clusters • Number of eligible participants available • Number of clusters can be randomised at one time point to receive the intervention • Study duration • Feasibility & costs ($$$) www. clinicaltrialsalliance. org. au

Cluster randomised trials –schematics TIME Parallel cluster RCT Intervention (A) Control (B) Stepped Wedge CRT (no switch back) www. clinicaltrialsalliance. org. au

Sample size calculation for CRT • Individuals that attend the same MHCSS (cluster) are more alike than individuals that attend other MHCSS (cluster) • Intra-cluster correlation (ICC; within cluster and between cluster variance) • Study efficiency/power decreases with increasing ICC and/or cluster size (compared to an individually randomised trial) www. clinicaltrialsalliance. org. au

Sample size calculation for CRT • Individuals that attend the same MHCSS (cluster) are more alike than individuals that attend other MHCSS (cluster) • Intra-cluster correlation (ICC; within cluster and between cluster variance) • Study efficiency/power decreases with increasing ICC and/or cluster size (compared to an individually randomised trial) Power calculations for a CRT with 9 clusters 270 participants (= 30 x 9 clusters) Effect size ICC Cluster size Number of clusters Power 0. 35 SD 0. 1 30 9 30% 5% significance level for 2 sided test www. clinicaltrialsalliance. org. au

Sample size calculation for CRT • Individuals that attend the same MHCSS (cluster) are more alike than individuals that attend other MHCSS (cluster) • Intra-cluster correlation (ICC; within cluster and between cluster variance) • Study efficiency/power decreases with increasing ICC and/or cluster size (compared to an individually randomised trial) Power calculations for a CRT with 9 clusters 270 participants (= 30 x 9 clusters) For 80% power need 34 clusters in total 1020 participants (= 30 x 34 clusters) Effect size ICC Cluster size Number of clusters Power 0. 35 SD 0. 1 30 9 30% 5% significance level for 2 sided test www. clinicaltrialsalliance. org. au

Sample size for SW-CRT with cross-sectional samples SW-CRT typically require fewer clusters than standard parallel CRT Uses both within cluster and between cluster variance to estimate the intervention effect – Relatively insensitive to the correlation of individuals belonging to the same cluster (intracluster correlation) Increase in power/study efficiency as number of measurements increase for a fixed number of clusters Assumes that the full effect of the intervention occurs at the same time interval when intervention is introduced 1 – Delay of intervention effect reduces study power – Increase power by adding additional measurement periods 1. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemporary Clinical Trials 2007 2. Hooper R, Teerenstra S, de Hoop E, Eldridge S. Sample size calculation for stepped wedge and other longitudinal cluster randomised trials. Statistics in www. clinicaltrialsalliance. org. au Medicine 2016 3. Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Sample size calculation for a stepped wedge trial. Trials. 2015

Example: CORE study • Monte-Carlo simulations for SW-CRT with cross-sectional and cohort design (closed or open) • Simulation program in R using Shiny (Dr Tibor Schuster, Mc. Gill University) Power calculations for a SW-CRT with 9 clusters and 3 steps 270 participants (= 30 x 9 clusters) www. clinicaltrialsalliance. org. au

Analytical methods for SW-CRTs • • • Published SW-CRTs use many different methods No consensus, active area of methodological research Pre-specify analytical approach One approach is to use mixed-effects regression Analysis should allow for – Correlation structures • Within cluster and between cluster variance (intra-cluster correlation) • Within-individual correlation (repeated measures) – Confounding by time • Study arm is time variant • Changes in the intervention effect may be due to changes in time e. g. patients may recover or deteriorate with time regardless of the intervention phase www. clinicaltrialsalliance. org. au

Challenges of the SW-CRT • Longer trial duration – May require a longer time to complete, esp. if intervention takes a long time to implement – Best when outcome is almost immediate after the intervention • Complex intervention – may take time for the intervention to be embedded & change the outcomes – Transition periods can be incorporated – but may further extend study duration, making the SW design impractical • Multiple data collection points – best if using routinely collected data – Repeated measures may increase burden on participants (cohort) if number steps in large • Risk of contamination between intervention participants and those waiting for intervention • All clusters need to be recruited prior to randomisation www. clinicaltrialsalliance. org. au

Summary Debate around the level of evidence provided by SW-CRT; need good rationale for its use 1, 2 Where possible, choose a parallel-CRT over the SW-CRT – More complex to design and analyse – Increased risk of bias, may impact the strength of evidence – Increases the length of the trial, particularly when many steps involved and implementation of intervention in long Registry-based trials may mitigate risk of bias and data collection burden, but has other challenges (eg. data quality) 1. Hemming K, Taljaard M, Mc. Kenzie J et al: Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with www. clinicaltrialsalliance. org. au explanation and elaboration. BMJ 2018 2. Hemming K, Taljaard M, . Reflection on modern methods: when is a stepped-wedge cluster randomized trial a good study design choice? International Journal of Epidemiology 2020

Thank you www. clinicaltrialsalliance. org. au