Randomised Controlled Trials in the Social Sciences Challenges

  • Slides: 41
Download presentation
Randomised Controlled Trials in the Social Sciences: Challenges and Prospects Assessing Nonpharmacological treatments Challenges

Randomised Controlled Trials in the Social Sciences: Challenges and Prospects Assessing Nonpharmacological treatments Challenges and reporting guidelines for Randomised Controlled Trials I. Boutron Dept of Epidemiology, Biostatistics and Clinical Research 12/1/2020 1

Overview • Specific methodological issues when assessing Nonpharmacological treatments • Quality tools • Reporting

Overview • Specific methodological issues when assessing Nonpharmacological treatments • Quality tools • Reporting guidelines 2

Overview • Specific methodological issues when assessing Nonpharmacological treatments • Quality tools • Reporting

Overview • Specific methodological issues when assessing Nonpharmacological treatments • Quality tools • Reporting guidelines 3

Different types of NPT Non-pharmacological treatments (NPT) take in many different treatments Therapist-dependent interventions

Different types of NPT Non-pharmacological treatments (NPT) take in many different treatments Therapist-dependent interventions - surgery Participative interventions - rehabilitation - technical operation (arthroscopy, …) - education Non implantable devices - orthopedic braces or orthosis - laser treatment

Assessing Nonpharmacological treatments • Randomized controlled trials (RCT) – Gold standard of therapeutic evaluation

Assessing Nonpharmacological treatments • Randomized controlled trials (RCT) – Gold standard of therapeutic evaluation • The design, conduct and reporting of RCTs aim at providing valid results • Standard of the methodology of RCTs has been developed in the context of pharmacological treatments (PT) • Specific methodological issues when assessing NPTs – – Blinding Placebo Complexity of treatments Care providers’ influence 5

Are NPT trials specific? Quality of trials assessing PT and NPT of hip and

Are NPT trials specific? Quality of trials assessing PT and NPT of hip and knee osteoarthritis published in high impact factor journals (1992 -2002)* A 6 5 4 3 2 1 0 -1 B PT NPT 6 5 4 3 2 1 0 -1 A. PT (n=60) versus NPT (n=50) B. Comparison (1) 1 Jadad scale * Boutron, Tubach, Giraudeau, Ravaud Jama, 2003 2 Jadad scale 3 4 (2) (3) (4) Oral drug, intra-venous, intramuscular (n = 46) Intra-articular injection (n = 14) Surgery, arthroscopy, joint lavage (n = 23) Rehabilitation, acupuncture, education(n = 27) 6

Are NPT trials specific? The feasibility of blinding * Boutron I, Tubach F, Giraudeau

Are NPT trials specific? The feasibility of blinding * Boutron I, Tubach F, Giraudeau B, Ravaud P, JCE, 2003 7

Are NPT trials specific? The success of blinding 8

Are NPT trials specific? The success of blinding 8

Are NPT trials specific? Placebo PT Surgery Rehabilitation, education Placebo Matching placebo Simulated procedure

Are NPT trials specific? Placebo PT Surgery Rehabilitation, education Placebo Matching placebo Simulated procedure Sham intervention Example Saline solution in identical syringes Sham arthroscopy* Sham education program with similar frequency and duration of sessions but with different content ** - Ethical issue False negative Issue * Moseley et al. , NEJM 2002 **Edworthy et al. J Rheumatol 1999 9

Are NPT trials specific? Placebo Paterson, C. et al. BMJ 2005 10

Are NPT trials specific? Placebo Paterson, C. et al. BMJ 2005 10

Are NPT trials specific? Placebo • Externally clearly different placebo – Physiotherapy for knee

Are NPT trials specific? Placebo • Externally clearly different placebo – Physiotherapy for knee OA* • Placebo treatment consisted of sham ultrasonography – Assessment of lifestyle advice** • Placebo treatment consisted of a syrup placebo *Bennell, Ann Rheum Dis, 2005 **Spigt, JCE, 2005 11

Are NPT trials specific? The methods of blinding – Full blinding Study cohort: 145

Are NPT trials specific? The methods of blinding – Full blinding Study cohort: 145 non-pharmacological trials (2004)* • Full blinding attempted • externally identical placebo/control • Unblinded care providers not involved in the subsequent patient care • Example – Transplantation of embryonic dopamine neurons** • four twist-drill holes made through the frontal bone after local anesthesia • the dura mater was not penetrated * Boutron, Ravaud, Submitted ** Freed, NEJM, 2001 12

Are Non pharmacological trials specific? The methods of blinding – Partial blinding • Partial

Are Non pharmacological trials specific? The methods of blinding – Partial blinding • Partial blinding attempted (blind trial hypothesis) – Externally clearly different placebo / control – Manipulation of information • Patients not informed of the existence of a placebo • Patients not aware of the nature of the placebo • Confidence in treatments sometimes tested 13

Are Non pharmacological trials specific? The methods of blinding – Partial blinding • Zelen

Are Non pharmacological trials specific? The methods of blinding – Partial blinding • Zelen design or modified zelen design – Usual care compared to a complex, physical therapy-based intervention for patello-femoral joint osteoarthritis of the knee • 1) Researchers invited patients to participate in a cohort. • 2) Randomization • 3) Patients randomized to the intervention arm informed that they would receive the experimental treatment and signed a second consent form Quilty, J Rheumatol. 2003 14

Are Non pharmacological trials specific? The methods of blinding – Outcome assessors • Blinding

Are Non pharmacological trials specific? The methods of blinding – Outcome assessors • Blinding of outcome assessors – Centralized assessment (video, audiotape or photography ) – Outcome assessors blinded of study hypothesis • To assess the analgesic effect of breast feeding in term neonates compared to mothers’ arms, pacifiers, placebo (i. e. , sterile water) or glucose – Videotape of the children during the painful procedure – Two specially trained observers independently assessed the recordings using a specific scale. – Observers were blinded to the purpose of the study and had been told that they were assessing agreement of their scores in different situations – Patients reported outcomes • Blinding is impossible if patients cannot be blinded Carbajal R, Bmj. 2003 15

Are NPT trials specific? Complex intervention • Several components – Rehabilitation : exercises, drugs,

Are NPT trials specific? Complex intervention • Several components – Rehabilitation : exercises, drugs, education etc • Description of the intervention – Quantitative data • Number of sessions, timing of each session, duration of each session – Qualitative data • Content of each session, how it is delivered, supervision, content of information exchanged etc • Standardization procedure – Specific training – Quality control procedures • Potential gap between the intended intervention (as described in the protocol ) and the actual administered intervention • Barrier for systematic reviews 16

Are NPT trials specific? Influence of care providers • Systematic review of the surgical

Are NPT trials specific? Influence of care providers • Systematic review of the surgical and medical volumeoutcome literature (Jan 1, 1980 - Dec 31, 2000)* – 71% of hospital volume-outcomes studies positive (88/124) – 70% of physician volume-outcomes studies positive (31/44) – No studies showed the opposite relationship – Relationship strongest for high risk/rare surgeries: • Pancreas/esophagus cancer, pediatric cardiac surgery • NNT at high v. low volume provider: 7 to 11 – Much more modest volume-outcome effect for common procedures (CABG, CEA, PTCA, breast/colon cancer) • NNT: 62 to 500 *Halm et al, Ann Intern Med 2002 17

Are NPT trials specific? • Specific issues in assessing NPT – Blinding – Placebo

Are NPT trials specific? • Specific issues in assessing NPT – Blinding – Placebo – Complexity of treatments – Healthcare providers’ influence • Need of specific standards 18

Overview • Specific methodological issues when assessing Nonpharmacological treatment • Quality tools • Reporting

Overview • Specific methodological issues when assessing Nonpharmacological treatment • Quality tools • Reporting guidelines 19

Quality tools Why? • Assessing the quality of reports of trials is particularly important

Quality tools Why? • Assessing the quality of reports of trials is particularly important – clinicians’ critical appraisal of healthcare literature – systematic reviews • Accurate estimates of the treatment effect 20

Quality tools Why? • Several quality tools • Few quality tools were developped according

Quality tools Why? • Several quality tools • Few quality tools were developped according to scientific standards – Jadad scale* – Delphi list** • Validated quality tools were mainly developped in the context of PT – Importance of blinding – No item related to care providers and the complexity of the intervention ** Jadad, Control Clin Trials, 1996 * Verhagen, JCE, 1998 21

Development of a specific checklist The example of CLEAR NPT • Quality: Internal validity

Development of a specific checklist The example of CLEAR NPT • Quality: Internal validity • Delphi consensus method • Selection of items • From existing checklist or scales • From specific items • From the Collaborative Review Groups of the Cochrane Collaboration recommandations • Interviews of clinicians • Experts • Members of Collaborative Review Groups of the Cochrane Collaboration (n=41) • Clinicians involved in RCTs assessing NPTs (n=58) • Methodologists, epidemiologists (n=55) – 55 experts participated(36%) Boutron & Ravaud, JCE, 200522

Initial checklist 38 items Decision of the steering comittee Selection of items with a

Initial checklist 38 items Decision of the steering comittee Selection of items with a score of 8/9 for more than 40% of experts 1 st Round 54 experts/55 Checklist 21 items 2 nd Round 46 experts/55 Decision of the steering comittee Selection of items With a score of 9/9 for more than 40% of experts Checklist 11 items Decision of the steering comittee Sélection of all items Addition of 2 items for randomisation Addition of 1 item for the intention-to-treat analyses 3 rd Round 49 experts/55 Final Checklist 10 items 5 sub items 23

CLEAR NPT • Was the generation of allocation sequences adequate? • Was the treatment

CLEAR NPT • Was the generation of allocation sequences adequate? • Was the treatment allocation concealed? • Were the main outcomes analyzed according to the intention-to-treat principle? • Were details of the intervention administered to each group made available? • Were care providers’ experience or skill in each arm appropriate? • Was participant (ie, patients) adherence assessed quantitatively? • Was the follow-up schedule the same in each group? 24

CLEAR NPT • Were participants adequately blinded? • Were care providers or persons caring

CLEAR NPT • Were participants adequately blinded? • Were care providers or persons caring for the participants adequately blinded? – If care providers and or participants were not adequately blinded • Were all other treatments and care (ie, co-interventions) the same in each randomized group? • Were withdrawals and lost to follow-up the same in each randomized group? • Were outcome assessors adequately blinded to assess the primary outcomes? – If outcome assessors were not adequately blinded, • Were specific methods used to avoid ascertainment bias (systematic differences in outcome assessment) 25

CLEAR NPT • Adequate assessment of trials quality • Improvement of critical appraisal of

CLEAR NPT • Adequate assessment of trials quality • Improvement of critical appraisal of medical litterature • Improvement of the quality of systematic reviews 26

Overview • Specific methodological issues when assessing Nonpharmacological treatment • Quality tools • Reporting

Overview • Specific methodological issues when assessing Nonpharmacological treatment • Quality tools • Reporting guidelines 27

Reporting guidelines Why? • Critical appraisal of the quality of clinical trials is possible

Reporting guidelines Why? • Critical appraisal of the quality of clinical trials is possible only if the design, conduct, and analysis of RCTs are thoroughly and accurately described in published articles • Evidence of incomplete and inadequate reporting 28

Reporting guidelines 29

Reporting guidelines 29

30

30

Reporting guidelines 31

Reporting guidelines 31

Reporting guidelines Dissemination • Publication – 11 journals – Over than 1000 citations •

Reporting guidelines Dissemination • Publication – 11 journals – Over than 1000 citations • Editors endorsements including the ICMJE (International Committee for Medical Journal Editors) – instructions to authors – submission check-list • Website – www. consort-statement. org 32

Reporting guidelines Dissemination • Various extensions of the CONSORT Statement since the original version

Reporting guidelines Dissemination • Various extensions of the CONSORT Statement since the original version (parallel arm, superiority, efficacy) • • Cluster RCT Harm Herbal therapy Non inferiority trials • Several other extensions are in progress • 2 x 2 factorial design 33

Reporting guidelines for NPT trials Why? • 119 RCTs published in 1998 and 1999

Reporting guidelines for NPT trials Why? • 119 RCTs published in 1998 and 1999 in three major cardiothoracic journals (Annals of Thoracic Surgery, European Journal of Cardiothoracic Surgery, The Journal of Thoracic and Cardiovascular Surgery) • 70 % of RCTs fulfilled less than half of the CONSORT criteria Anyannu, Eur J of Cardio-thoracic Surgery, 2004 34

Reporting guidelines for NPT trials Why? Study of trials assessing PT and NPT of

Reporting guidelines for NPT trials Why? Study of trials assessing PT and NPT of hip and knee osteoarthritis (1992 -2002) When blinding was judge possible, blinding was less often reported in NPT trials NPT PT Blinding of patients 46% 98% Blinding of healthcare providers 43% 83% Blinding of outcome assessors 72% 98% * Boutron & Ravaud, Jama, 2003 35

Reporting guidelines for NPT trials Why? Study of surgical papers (n = 158) published

Reporting guidelines for NPT trials Why? Study of surgical papers (n = 158) published in 2004* • Reporting of details of the intended intervention – – Surgical procedure Pre-operative care Anesthesia Post-operative care 87 % 15 % 35 % 49 % • Surgeons – Selection criteria for surgeon – Number of interventions performed by surgeon – Number of surgeons involved * Jacquier I, Boutron I, Ravaud P, Ann. Surgery 40 % 11 % 33 % 36

Reporting guidelines for NPT trials Why? Study of rehabilitation papers (n = 171) published

Reporting guidelines for NPT trials Why? Study of rehabilitation papers (n = 171) published between 1997 -1998* • Timing of the intervention • Description of the intervention • Intervention’s adherence * Dijkers at al. Am J Phys Med Rehabil, 2002. 68 % 50 % 34% 37

Reporting guidelines Development of the extention of the CONSORT to NPT Steering committee –

Reporting guidelines Development of the extention of the CONSORT to NPT Steering committee – D. Moher – P. Ravaud – I. Boutron Selection of experts – Editors – Clinicians – Methodologists Two stages method – Preliminary survey – Consensus meeting 38

Preliminary survey • Objective – To identify items to be discussed based the experts’

Preliminary survey • Objective – To identify items to be discussed based the experts’ opinion • Methods – All items of the CONSORT checklist – 7 specific questions on key issues were added – Items were selected for discussion if more than 1/3 of the experts answered that the item should be modified or another item should be added 39

Consensus meeting • 3 days – February 8 -10 th 2006, Paris • 30

Consensus meeting • 3 days – February 8 -10 th 2006, Paris • 30 experts • Agenda – Presentations – Discussions – Consensus • Manuscript in process 40

Conclusions • Specific issues in assessing NPT – – Placebo Blinding Complexity of the

Conclusions • Specific issues in assessing NPT – – Placebo Blinding Complexity of the intervention Care providers’ influence • Specific guidelines – Specific quality tools • CLEAR NPT – Specific reporting guidelines • Development of an extension of the CONSORT for Non. Pharmacological treatment (CONSORT meeting, Paris, 2006) 41