Handling clinical trials data from existing and new

Handling clinical trials data from existing and new electronic systems – Expectations from DKMA DADM Conference 5 October 2020 Ib Alstrup, Medicines Inspector Gx. P IT, Danish Medicines Agency

Questions received Decommisioning • How should sponsors document the system functionality? Are PDF files including the data and pictures of how they were gathered (blank CRFs) sufficient? • Investigators access to data. Is it fine to revoke the access when sponsor have received acceptance of receipt of the hard drive containing the data from sites? • Audit trail. Is it fine to have this as PDF file only (hard to search for specific data changes)? Ongoing: • How much review of the audit trail is expected from the regulators? (Please provide examples of which kind of data and reviews you will expect us to look at) Is it sufficient to look for trends ‘by sites’ and ‘by data items’? New devices/ways of collecting data: • Before starting working with a new system, what QC is expected to the audit trail functionality? How is this dependent of whether the system is seen as SAAS (Software As A Service) • BYOD (Bring Your Own Device). What to focus on when documenting why data is collected this way? – When is better patient compliance outweighing the risk for ‘wrong’ data (registered by someone else or local setting changed)? • Automatic data collection tools (e. g. activity trackers). What to focus on when documenting which data are used? In general a data mining is taking place and the algorithm is not known to the sponsor. Do we need to archive the raw data, or could the data mining result be defined as raw data? • Any new device. What to focus on when documenting why this device is used. What kind of mitigations are expected to see? 2 • How are the documentation dependent on the way sponsors use the software? SAAS or sponsor ‘owned’ – and in what category do we see the mobile apps? 4. OKTOBER 2020

Question 1: Documenting system functionality and data (at decommisioning): How should sponsors document the system functionality? • URS, incl. detailed layout of screens (e. g. VAS scales) where applicable • Corresponding qualification documentation (made prior to start of study) • Study specific configuration specification, edit checks and user acceptance test (UAT) • User guides, SOPs and quality records (change requests, deviations etc. ) • Vendor documentation where applicable Are PDF files including the data and pictures of how they were gathered (blank CRFs) sufficient? • No, PDF of CRFs would not sufficiently describe system functionality (e. g. edit checks) 3 4. OKTOBER 2020

Question 2: Revoking sponsor’s access (at decommisioning): Investigators access to data. Is it fine to revoke the access when sponsor have received acceptance of receipt of the hard drive containing the data from sites? • Yes, this would normally be accepted but investigator should have acknowledged the data 4 4. OKTOBER 2020

Question 3: Audit trail as PDF (at decommisioning): Audit trail. Is it fine to have this as PDF file only (hard to search for specific data changes)? • Exporting audit trail to Excel implies obvious risk to data integrity (it may be changed) • Exporting audit trail to PDF implies loss of dynamic properties (e. g. conditional search) Prefered • Keep data in qualified application (database) • Migrate data to another qualified system Work arounds may include (suggestion) • Qualified Excel export functionality which saves checksum of Excel (kept safe) • Export to table in PDF which allowes for easy re-export to Excel 5 4. OKTOBER 2020

Question 4: Audit trail review (ongoing): How much review of the audit trail is expected from the regulators? (Please provide examples of which kind of data and reviews you will expect us to look at)? • Late changes and changes made by unusual personel or at unusual date and time • Changes made by particular sites or individuals • Changes with inadequate reason (compromises data integrity) • Changes made following incomplete or missing data change request (DCR) • Changes to critical data (e. g. primary effect parameters) • Changes made after edit checks fired (e. g. inclusion criteria not met) • It is not requested to review the obvious (e. g. first time entry of weight) Is it sufficient to look for trends ‘by sites’ and ‘by data items’? • Not static process, continuously evaluated and improved if applicable 6 4. OKTOBER 2020

Findings related to audit trail review There was no procedure for audit trail reviews and no documentation that such reviews had been conducted for the ███ [e. CRF] and ███ [CDMS] systems. There was no procedure for audit trail review and no documentation that audit trails for the e. PRO system used to collect primary efficacy data, had been reviewed. There was no documentation that sponsors CRO, ███, had procedures for or had conducted audit trail reviews on any of the two inspected ███ [e. COA] and ███ [e. CRF] systems (100%). There was no procedure or practice for audit trail review for 2 out of 2 (100%) inspected systems [used] within GCP. 7 4. OKTOBER 2020

Question 5: Qualification of audit trail functionality (new devices/ways of collecting data): Before starting working with a new system, what QC is expected of the audit trail functionality? • The audit trail functionality should be qualified (like any other critical functionality) • Before starting the study (put into use), the qualification should demonstrate that system functionality meets system requirements and regulatory expectations (e. g. audit trail) • Qualification of autit trail should demonstrate that manual user interactions are correctly and automatically logged (who, what, when and why) and cannot be modified by users How is this dependent of whether the system is seen as SAAS (Software As A Service)? • No difference, someone should qualify the system and this should be documented during regulatory inspection 8 4. OKTOBER 2020

Finding related to qualification of audit trail functionality There was insufficient qualification for one out of 11 (9%) inspected requirements from the URS for Electronic Data Capture (EDC), ███, internal no. ███, while qualification of the remaining requirements was found to be satisfactory: The system was provided as software as a service (Saa. S), but except for one of the selected requirements (first) where the sponsor had relied on vendor qualification documentation, the sponsor had conducted their own qualification. • URS-███ “A user must be able to review a response to an automatically generated query and accept the response and have the query disappear from the screen-but tracked in the audit trail. ” Test documentation was insufficient as it only showed test of an automatic query (edit check), it did not show the very important element, that the firing of an automatic query would be captured by the audit trail. 9 4. OKTOBER 2020

Question 6: BYOD (new devices/ways of collecting data): BYOD (Bring Your Own Device). What to focus on when documenting why data is collected this way? • Risk management (i. e. assessment and mitigation), incl. at least the following parameters • Access control (e. g. sharing of passwords and password management services) • Possibility to set date, time and timezone, inactivity timeout etc. • Device-specific qualification of apps and handling of IT security • Category of collected data (i. e. primary effect parameters or supporting data) When is better patient compliance outweighing the risk for ‘wrong’ data (registered by someone else or local setting changed)? • It is never more important that the patient has reported data, than that it is correct • Or, when data integrity is not important 10 4. OKTOBER 2020

Question 7: Data collection tools and raw data (new devices/ways of collecting data): Automatic data collection tools (e. g. activity trackers). What to focus on when documenting which data are used? • Risk management and focus areas as for BYOD (provided used for critical data) In general a data mining is taking place and the algorithm is not known to the sponsor. Do we need to archive the raw data, or could the data mining result be defined as raw data? • No yet seen in inspection, still needs discussion in EMA GCP Inspector’s Working Group 11 4. OKTOBER 2020

Question 8: Reason for using a device (new devices/ways of collecting data): Any new device. What to focus on when documenting why this device is used. What kind of mitigations are expected to see? • Risk management and focus areas as for BYOD • Mitigation should effectively bring risks down to an acceptable level • Someone must have qualified the device and document this during regulatory inspection (if not just for exploratory use) 12 4. OKTOBER 2020

Question 9: Saa. S and mobile apps (new devices/ways of collecting data): How are the documentation dependent on the way sponsors use the software? SAAS or sponsor ‘owned’ – and in what category do we see the mobile apps? • No difference between systems provided as Saa. S and locally installed systems • The sponsor maintains full responsibility and both systems should be qualified and operated (incl. IT security) in a manner which ensures data integrity • Documentation for the above should be available during regulatory inspection • See EMA GCP Q&A no. 9 https: //www. ema. europa. eu/en/human-regulatory/research-development/compliance/good -clinical-practice/qa-good-clinical-practice-gcp 13 4. OKTOBER 2020

Soon to come from EMA GCP Inspector’s Working Group (for public consultation) Guideline on Computerised Systems and Electronic Data in Clinical Trials 14 4. OKTOBER 2020

Thanks for your attention For questions: Ib Alstrup, Medicines Inspector Gx. P IT, Danish Medicines Agency ibal@dkma. dk, www. linkedin. com/in/ib-alstrup-baa 2542