BRIEF RECOMENDATIONS FOR DESIGNING YOUR RANDOMIZED CONTROLLED TRIAL
(BRIEF) RECOMENDATIONS FOR DESIGNING YOUR RANDOMIZED CONTROLLED TRIAL (RCT) Dr. Matthew Keough August 8 th, 2018 Summer School
WHAT IS AN RCT? A randomized controlled trial (RCT) is an experiment in which investigators randomly assign eligible human research participants or other units of study (e. g. , classrooms, clinics, playgrounds) into groups to receive or not receive one or more interventions that are being compared. The results are analyzed by comparing outcomes in the groups. (CIHR, 2018) A scientific experiment that tests a new treatment, while simultaneously tries to reduce bias (Kraemer & Wilson, 2002)
Psychological Perspective
Meta. Analysis Strongest RCTs Cohort studies Case-control studies Case series, case reports Editorials, expert opinion Weakest
RECOMMENDATION #1 1. MAKE SURE THAT YOUR TREATMENT IS EMPIRICALLY-INFORMED • Are you the first one to tackle this problem? • What has been done already? • Is there a “best practice” treatment that you can improve on?
RECOMMENDATION #2 2. BE TRANSPARENT • Pre-register your trial protocol • Do what you say you’ll do • Do not deviate from your original plans
QUESTION: HOW DO NULL RESULTS RELATE TO REGISTRATION?
RECOMMENDATION #3 3. DESIGN WELL AND MEASURE • Who is in your target population? • Who will you allow in? • What groups will you have? • What symptoms/variables will you measure and how often?
WHO IS IN YOUR TARGET POPULATION AND WHO WILL YOU LET IN? • Traditional method: recruit ”purest” groups possible (i. e. , maximize internal validity) • Easier said than done (especially in psychology) • Recent method: allow for more “complex groups” but just measure and model it statistically (i. e. , maximize external validity)
QUESTION: WHY DO YOU THINK HAVING A HETEROGENEOUS SAMPLE IS BENEFICIAL?
SIDENOTE: EFFICACY VS. EFFECTIVENESS Efficacy: testing if your treatment works “in the lab” or under ideal, controlled conditions Specific interventions Internal validity Strict methodology Hunsley, Elliot, & Therrien, 2013; Nathan, Stuart, & Dolan, 2000
SIDENOTE: EFFICACY VS. EFFECTIVENESS Effectiveness: testing if your treatment works “in the real world” or under less controlled conditions Feasibility, real-world situations External validity More open methodology Hunsley, Elliot, & Therrien, 2013; Nathan, Stuart, & Dolan, 2000
WHAT GROUPS WILL YOU HAVE? • Clinical trials examine how something performs compared to another (Nathan, Stuart, & Dolan, 2000; Parloff, 1986) 1. Waitlist control: = passive 2. ‘Standard practice’ control = receiving commonly accepted treatment (i. e. , TAU) 3. Placebo control group
QUESTION: HOW DOES THE TYPE OF CONTROL GROUP IMPACT INTERPRETATION?
METHODOLOGY: BLINDING Blinding is when the patient/researcher or both are unaware of the condition that they are in • Psychotherapy research most often has patient single-blinding • Pro: Eliminates bias • Con: Very difficult to have double-blinding (Nathan, Stuart, & Dolan, 2000)
QUESTION: WHY DO YOU THINK IT IS DIFFICULT TO HAVE COMPLETE BLINDING?
What symptoms/variables will you measure and how often? • Need to specify a primary outcome • Specify secondary outcomes (and you should!) • Make sure all measures are reliable (and ideally have been validated in your target population) • Measure moderators and mediators as well!!
EVALUATING OUTCOMES: MODERATION AND MEDIATION • Moderators specify for whom and/or under what conditions treatment works (Kraemer & Wilson, 2002) • Identify subpopulations with possibly different causal mechanisms and who may benefit most • Meditators identify possible mechanisms through which a treatment might achieve its effects • The so-called “why”
QUESTION: WHAT IS THE RELEVANCE OF MODERATORS AND MEDIATORS FOR EVALUATING OUTCOMES?
RECOMMENDATION #4 4. DECIDE HOW YOU WILL HANDLE MISSING DATA • How will you handle drop-outs? • Should you include them? Or exclude them?
QUESTION: WHY DO DROP-OUTS POSE A PROBLEM FOR AN RCT?
DECIDE HOW YOU WILL HANDLE MISSING DATA • INTENTION TO TREAT • Include all randomized participants • Impute data OR use full information methods • Depends on “pattern of missingness” or lack thereof • COMPLETE CASE ANALYSIS • Not the best method BUT, can be unbiased if missingness is random
- Slides: 25