Transforming Data Quality Using Machine Learning SCOPE February

Transforming Data Quality Using Machine Learning SCOPE, February 2019

Stacey Yount Vice President, Product Clinical Trial Management Medidata Solutions © 2019 Medidata Solutions,

Objectives • Assess current state data quality oversight efficiency and effectiveness • Demonstrate why machine learning provides opportunities for transformation • Review case studies that highlight the impact of ML in the data review cycle © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 3

Data Feeds Are Complex And Increasing CTMS Mobile health BUDGETING EHR/ EMR Biomarkers SDR PROTOCOL DESIGN LIMS CTMS e. PRO WEB SERVICES SAFETY Labs Listings PAYMENTS SAS extracts IRT SENSORS CODING IMAGING EDC BATCH UPLOAD APPS Claims e. Diaries SQM © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 4

Processes and Technology Study Milestones Investigator Sites Inefficient edit checks and manual data review with 7. 3% of submissions failing 1 st attempt due to data issues Database locks averaging between 30+ days due to issues uncovered late in study lifecycle Increased frustration by sites and clinical teams with increased queries near database lock Organizational Value Employee Engagement Concerns voiced by Statistics organizations regarding cleanliness of data and use of Stats resources for cleaning Negative perception due to inefficient processes, impact to study milestones, and lack of insights prior to final reviews © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 5

Centralized Statistical Analytics • Known and unknown risk discovery through the use of statistical, machine learning analytics • Data quality and site performance • Single place to ensure logical and statistical data quality across all of your “monitoring” functions © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 6

Data Management Reimagined Robust Edit Checks & Standards Integrated Quality & Risk Management Process Risk Oversight Team CSA Reviews Earlier Stats Reviews Risk Evaluation Outcome Risk Communication Pathway © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 7

Technology Enabled Convergence RBM and advanced analytics create opportunities for operational convergence © 2019

Hurdles • • Learning curve with new technology Significant as-is to-be differences in processes Lack of industry best practices Misalignment with current skill set © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 9

Best Practices • Design a new process instead of changing an existing process • Adjust implementation to account for organizational plans (e. g. , size, structural changes) • Gain cross-functional alignment to process and tech changes • Leverage a team approach to oversight of data quality and site performance management © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 10

CSA Case Studies © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 11

Transformation 5 day database locks © 2019 Medidata Solutions, Inc. – Proprietary and Confidential

>25% of data quality issues found across recent studies had potential to delay drug

Medidata CSA impact on submission Sponsor A Remediation FDA Approval Q 1’ 18 Launch Centralized Statistical Analytics Extra analyses Sponsor B FDA Letter Extra analyses Remediation Delay © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 14

Supplement © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 15

Illustration • “My medical record says my systolic BP was 180 in Jan 2017. ” Is this right or wrong? • What can you use from my medical record to help answer the question? © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 16

Illustration • My medical record says my systolic BP was 180 in Jan 2017. Is this right or wrong? • What can you use from my medical record to help answer the question? • Systolic BP before/after • Diastolic BP measurement • Medical history or hypertension • Concurrent hypertension adverse event reported • Con Med Hypertension Drug • Risk Factors (Age/Sex/etc. ) • After reviewing all of my data, could you answer the question with reasonable confidence? © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 17

Illustration • How could we use medical records of people with characteristics similar to mine and give an even more accurate, quantitative answer to whether the Systolic BP is an error? © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 18

Illustration • How could we use medical records of people like me with the list we came up with and give an even more accurate, quantitative answer to whether the Systolic BP is an error? • Normal variation of Systolic BP • Slope of Systolic BP vs. Diastolic BP relationship • Ranking of risk factors for relationship to Systolic BP Bottom Line: You can calculate the likelihood of data being wrong with higher confidence © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 19

$Current data review practices lead to inefficiency and sub-optimal data quality Significant fraction of$

Current data review practices lead to inefficiency and sub-optimal data quality Significant fraction of data quality issues cannot be identified by traditional data review Site performance is not equal; some are significantly worse Most sites make mistakes in specific areas (which may differ from site to site) Most EDC data entered by sites are correct, but errors can have significant impact on speed and success of study © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 20

Most common quality issues uncovered with statistical data cleaning Site inconsistency for unknown risks Site inconsistency for known risks Differences in adverse event reporting Inconsistencies in how sites evaluate or measure endpoints Inconsistencies in how sites follow the protocol Differences in the actions sites take with regard to an adverse event Potential misconduct Data inconsistency Sites that make up data out of neglect or forgetfulness Data that are impossible or highly unlikely due to data entry errors © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 21

Iterative value of process ● Risk Oversight Team provides cross-functional governance over data quality and site performance ● Technology-enabled process optimization provides early insights ● Iterative process provides feedback and insights into IQRM and oversight ● Risk Communication Pathways ensure clear understanding of risk Site Corrective Action Plan(s) Study Team Accountability Assignments Early Escalation of Safety and Efficacy Issues Portfolio Compound Risk Evaluation Expedited Database Lock Timelines © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 22

Processes and Technology Efficient iterative processes supported by ML-based technology; nearly doubling number of database locks YOY Mid-sized sponsor transforms clinical data management using Medidata CSA Study Milestones Database locks average 5 days for critical studies and 10 days for others with significantly fewer findings after LPLV Investigator Sites Improved sites, clinical team communications and involvement in review cycles; excited to participate in study milestones Organizational Value Statistics findings 0 to <2%; Positive organizational satisfaction CDM Job Satisfaction Empowered through early insights with increased confidence in data quality and completeness = high job satisfaction and retention © 2019 Medidata Solutions, Inc. – Proprietary and Confidential 23