Un debate entre la validez interna del Ensayo

Un debate entre la validez interna del Ensayo Clínico y la externa del RWD Confrontación o cohabitación Rodrigo Dienstmann

PUBMED search “Real World Data” and “Cancer” 450 400 300 250 200 150 100 50 19 20 09 20 99 19 89 19 79 0 19 citations 350

Booth et al, Nat Rev Clin Oncol 2019

Definition – Real World Data Real-world data (RWD) is defined as data derived from a number of sources that are associated with outcomes in a heterogeneous patient population in real-world settings. BIG and/or Thick data: registries, large cohort studies, electronic records… Important: not every retrospective observational (database) study is RWD!

Definition – Real World Data generalizable (externally valid) Scientific strength ) T d zi e l (RC m ria o d lt n Ra nica cli not generalizable (internally valid) ic l t a ia m tr g a al r P nic cli y e d v v i t tu ti stu c c s e l a p p n s na s o o o r tio t Pr vati e a R r v r e e s s b b o o

Definition – Real World Data Clin Pharmacol Ther 2012

Definition – Real World Data Sample: Trial Participants Sample: All Eligible Patients 3: r o Err nown s unk nown unk 2: r o Err wn s kno nown unk 1: e r o Err ct siz e eff Effect size estimate Sample: All Patients REAL WORLD TOTAL Error: Error 1 + Error 2 + Error 3

Definition – Real World Evidence

Proposed approach OLD EVIDENCE-BASED MEDICINE Trial NEW EVIDENCE-BASED MEDICINE Precision Medicine Analysis of treatment effects RWD Trial Unbiased analysis Policy Outcome If positive Regulatory Agencies, HTA, etc. Guidelines Information to Patients, etc.

Who cares about RWD? PERFORMANCE OF AN INTERVENTION IN REAL LIFE Physicians Patients Insights on populations not tested in RCTs, larger sample size or longer follow-up, comparison between competitors, side effect profile in real practice… Regulatory agencies Politicians Payers Realistic expectations of patients outcomes, comparative cost-effectiveness, use of healthcare resources, clinical practice trends, burden of the disease gaps in cancer care. . .

Three dimensions of RWD/RWE Transparency Healthcare Policy Care Research Education Healthcare professionals Healthcare Management Results-oriented Sustainability Learning hospital

Learning Hospital Structured clinical data + Manual curation = Standard + Machine = Learning analytics

FDA perspective RWE in the pre-market context. Estimate the “fitness” of RWE. Extend indications to populations not included in RCT.

EMA perspective

RWD is not a substitute of RCT Complementary Alternative? - close the gap efficacy-effectiveness - long-term benefit and tolerability - biomarker validation - rare populations - ethical issues (promising treatments without controls) - logistical issues (competing interests, etc. )

How to make a RWD study comparable to RCT? Statistical considerations: Population selection 1. Which patients? 2. How many patients? 3. Do I need a control group? 4. What is the most appropriate control group? Bias control 1. Selection bias 2. Immortal-time bias 3. Information bias/ recall bias 4. Confounding by indication

Selection bias Distortion in the estimate of the association between intervention and endpoint that results from how patients were selected to the study.

Immortal-time bias

Information bias Systematic measurement error or misclassification of subjects on one of more variables, either on the intervention or the endpoint. e i v r e Int r e w s a i b R l l a ec s a i b

Confounding by indication Outcome depends not only on intervention but also on reasons for the indication (physician choice). Treatment A or B Survival Age, sex, socioeconomic status, prior therapies, metastatic sites, tumor burden, comorbidities…

Statistical alternatives Stratified analysis (stratified HR): analysis of homogenous strata (but allows adjustment for only a few potential confounding factors and sometimes substantial sample size loss) Multivariable models (adjusted HR): allows simultaneous adjustment for many factors without sample size loss, including exposure to therapy as time-varying covariate (but clinical interpretation can be very complex) Propensity score (HR): calculate the chance of assigning a patient to a given treatment based on all known confounding factors using a logistic regression model (score 0 -1).

Propensity score (PS) In a RCT the PS is 0. 5 for all patients assigned to treatment A or B In a RWD study the PS is used to obtain homogeneous groups in treatment A or B

Propensity score (PS)

Prior therapies Metastatic sites

Propensity score (PS) matching

Concerns about RWD Inefficient way of answering clinical questions? • Data quality • EMR not organized to support research • Not easy to recognize confounding variables • Generation of incorrect and unreliable conclusions Solution? • Relaxed inclusion criteria in RCT (make them real-life!)

Good quality data and prospective design are critical! We can only adjust for known confounding factors but if clinical annotation is good and complete the external validity is high. But… even so it can yield biased analyses due to unmeasured differences between treatment groups. CAREFUL INTERPRETATION!

From RWD to RCT Small observational studies RCT RWD effectiveness studies RWD Help design RCT Understand natural history of disease Generate synthetic control arms

Future: impact of treatment sequence Lilly, Health. Core Hess et al, JMIR Res Protoc 2017

Future: Cost-effectiveness PHAROS – Population-based HAematological Registry for Observational Studies Blommestein et al, Pharmaco. Economics 2015

Future: telemedicine and patient as data donor

Future: coverage and payment decisions Software platforms specifically designed for RWD analyses can strengthen implementation quality and will improve transparency.

Value-based pricing: When the price of a drug is based on its measured benefits in RCT. Outcomes-based contracting: Agreement between manufacturers and payers, in which manufacturer is obliged to issue a refund or rebate is linked to how well therapy performs in the real-world.