BRIEF RECOMENDATIONS FOR DESIGNING YOUR RANDOMIZED CONTROLLED TRIAL

(BRIEF) RECOMENDATIONS FOR DESIGNING YOUR RANDOMIZED CONTROLLED TRIAL (RCT) Dr. Matthew Keough August 8 th, 2018 Summer School

WHAT IS AN RCT? A randomized controlled trial (RCT) is an experiment in which investigators randomly assign eligible human research participants or other units of study (e. g. , classrooms, clinics, playgrounds) into groups to receive or not receive one or more interventions that are being compared. The results are analyzed by comparing outcomes in the groups. (CIHR, 2018) A scientific experiment that tests a new treatment, while simultaneously tries to reduce bias (Kraemer & Wilson, 2002)

Psychological Perspective

Meta. Analysis Strongest RCTs Cohort studies Case-control studies Case series, case reports Editorials, expert opinion Weakest

RECOMMENDATION #1 1. MAKE SURE THAT YOUR TREATMENT IS EMPIRICALLY-INFORMED • Are you the first one to tackle this problem? • What has been done already? • Is there a “best practice” treatment that you can improve on?

RECOMMENDATION #2 2. BE TRANSPARENT • Pre-register your trial protocol • Do what you say you’ll do • Do not deviate from your original plans

QUESTION: HOW DO NULL RESULTS RELATE TO REGISTRATION?

RECOMMENDATION #3 3. DESIGN WELL AND MEASURE • Who is in your target population? • Who will you allow in? • What groups will you have? • What symptoms/variables will you measure and how often?

WHO IS IN YOUR TARGET POPULATION AND WHO WILL YOU LET IN? • Traditional method: recruit ”purest” groups possible (i. e. , maximize internal validity) • Easier said than done (especially in psychology) • Recent method: allow for more “complex groups” but just measure and model it statistically (i. e. , maximize external validity)

QUESTION: WHY DO YOU THINK HAVING A HETEROGENEOUS SAMPLE IS BENEFICIAL?

SIDENOTE: EFFICACY VS. EFFECTIVENESS Efficacy: testing if your treatment works “in the lab” or under ideal, controlled conditions Specific interventions Internal validity Strict methodology Hunsley, Elliot, & Therrien, 2013; Nathan, Stuart, & Dolan, 2000

SIDENOTE: EFFICACY VS. EFFECTIVENESS Effectiveness: testing if your treatment works “in the real world” or under less controlled conditions Feasibility, real-world situations External validity More open methodology Hunsley, Elliot, & Therrien, 2013; Nathan, Stuart, & Dolan, 2000

WHAT GROUPS WILL YOU HAVE? • Clinical trials examine how something performs compared to another (Nathan, Stuart, & Dolan, 2000; Parloff, 1986) 1. Waitlist control: = passive 2. ‘Standard practice’ control = receiving commonly accepted treatment (i. e. , TAU) 3. Placebo control group

QUESTION: HOW DOES THE TYPE OF CONTROL GROUP IMPACT INTERPRETATION?

METHODOLOGY: BLINDING Blinding is when the patient/researcher or both are unaware of the condition that they are in • Psychotherapy research most often has patient single-blinding • Pro: Eliminates bias • Con: Very difficult to have double-blinding (Nathan, Stuart, & Dolan, 2000)

QUESTION: WHY DO YOU THINK IT IS DIFFICULT TO HAVE COMPLETE BLINDING?

What symptoms/variables will you measure and how often? • Need to specify a primary outcome • Specify secondary outcomes (and you should!) • Make sure all measures are reliable (and ideally have been validated in your target population) • Measure moderators and mediators as well!!

EVALUATING OUTCOMES: MODERATION AND MEDIATION • Moderators specify for whom and/or under what conditions treatment works (Kraemer & Wilson, 2002) • Identify subpopulations with possibly different causal mechanisms and who may benefit most • Meditators identify possible mechanisms through which a treatment might achieve its effects • The so-called “why”

QUESTION: WHAT IS THE RELEVANCE OF MODERATORS AND MEDIATORS FOR EVALUATING OUTCOMES?

RECOMMENDATION #4 4. DECIDE HOW YOU WILL HANDLE MISSING DATA • How will you handle drop-outs? • Should you include them? Or exclude them?

QUESTION: WHY DO DROP-OUTS POSE A PROBLEM FOR AN RCT?

DECIDE HOW YOU WILL HANDLE MISSING DATA • INTENTION TO TREAT • Include all randomized participants • Impute data OR use full information methods • Depends on “pattern of missingness” or lack thereof • COMPLETE CASE ANALYSIS • Not the best method BUT, can be unbiased if missingness is random