FDA Regulatory Perspective Data Integrity Steve Wilson Dr

FDA Regulatory Perspective: Data Integrity Steve Wilson, Dr. P. H. , CAPT USPHS Deputy Director, Division of Biometrics II, CDER/FDA NIH Roadmap Program Feasibility of Integrating & Expanding Clinical Research Networks 4 th Steering Committee Meeting Friday, May 12, 2006 NIH Natcher Conference Center, Bethesda, MD

FDA’s Mission • The FDA is responsible for protecting the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, our nation’s food supply, cosmetics, and products that emit radiation. • The FDA is also responsible for advancing the public health by helping to speed innovations that make medicines and foods more effective, safer, and more affordable; and helping the public get the accurate, science-based information they need to use medicines and foods to improve their health.

Data Integrity • The quality of correctness, completeness, wholeness, soundness and compliance with the intention of the creators of the data. • It is achieved by preventing accidental or deliberate but unauthorized insertion, modification or destruction of data in a database. • Data integrity is one of the six fundamental components of information security. http: //www. pcmag. com/encyclopedia_term/0, 2542, t=data+integrity&i=40792, 00. asp

Data Integrity • 21 CFR 11 ( In final rule “integrity” is used 50 times!) • Electronic Submission • Gateway • EDR SOPs

21 CFR 11 -- Integrity For electronic records and submissions to have the same integrity as paper records, they must be developed, maintained, and used under circumstances that make it difficult for them to be inappropriately modified. Without these assurances, FDA’s objective of enabling electronic records and signatures to have standing equal to paper records and handwritten signatures, and to satisfy the requirements of existing statutes and regulations, cannot be met.

Data Quality Data are of high quality "if they are fit for their intended uses in operations, decision making and planning” (J. M. Juran) http: //en. wikipedia. org/wiki/Data_quality

FDA Regulatory Perspective: Data Quality Steve Wilson, Dr. P. H. , CAPT USPHS Deputy Director, Division of Biometrics II, CDER/FDA NIH Roadmap Program Feasibility of Integrating & Expanding Clinical Research Networks 4 th Steering Committee Meeting Friday, May 12, 2006 NIH Natcher Conference Center, Bethesda, MD

Disclaimer Views expressed in this presentation are those of the speaker and not, necessarily, of the Food and Drug Administration

Truth-in-Advertising • CDER-Centric View • Customer -- Not in Office of Compliance or ORA

Organization: FDA • Center for Biologics Evaluation and Research (CBER) • Center for Devices and Radiological Health (CDRH) • Center for Drug Evaluation and Research (CDER) • Center for Food Safety and Applied Nutrition (CFSAN) • Center for Veterinary Medicine (CVM) • National Center for Toxicological Research (NCTR) • Office of the Commissioner (OC) • Office of Regulatory Affairs (ORA)

Outline • Background • Data Quality – A Bimo perspective – A Review perspective • Changing Landscape – – The Critical Path Data standards & electronic submissions PRO and e. PRO Organizational Changes • Bimo Initiative • CDER’s new Office of Translational Science (OTS) • Connections/References

Science, Statistics and Experimental Design • Science is concerned with understanding variability in nature • Statistics is concerned with making decisions about nature in the presence of variability • Experimental design is concerned with reducing and controlling variability in ways which make statistical theory applicable to decisions about nature. Winer, et. al. , Statistical Principles in Experimental Design, New York, 1962, 1971, 1991

Data Quality • Guidance for Industry: Computerized Systems Used in Clinical Trials, April 1999 • Data should be (ALCOA) – attributable – legible – contemporaneous – original – accurate

Responsibility • The sponsor is responsible for implementing and maintaining quality assurance and quality control systems with written SOP’s to ensure that trials are conducted and data are generated. . . • . . . Quality control should be applied to each stage of data handling to ensure that all data are reliable and have been processed correctly… Section 5. 1, “Quality Assurance and Quality Control, ” Good Clinical Practice Consolidated Guideline -- ICH E 6

The Law Section 505(k)(2) of the Food, Drug, and Cosmetic Act mandates FDA shall have access to and copy and verify the required clinical study records. The Program The Bioresearch Monitoring (BIMO) Program was established in 1977 to verify the data submitted in support of marketing applications and to provide oversight of the conduct of studies with regulated products.

Ketek – The “Fraud” Word Infected Data – Fraud, Errors Taint Key Study of Widely Used Sanofi Drug – Despite Some Faked Results, FDA Approves Antibiotic; One Doctor's Cocaine Use – Company Defends Safety By ANNA WILDE MATHEWS Wall Street Journal, May 1, 2006; Page A 1 online. wsj. com/article_print/SB 114644463095840108. htm

Not the First Time (Never Just a Scientific Question) • Halcion • Tamoxifen • Fiddes

Our Responsibility for the Quality of Clinical Trials Data Trust, but Verify

Worrying About Data Quality: NDA Review • Erroneous values • Missing values • Imputation • Adjudication • Deviations from protocol • “Unconscious Bias” • Fraud

Enforcer Colleague The Terminator Mr. Rogers

Programs That Worry About Data Integrity/Quality • BIMO • REVIEW

Data Integrity/Quality A BIMO Perspective • FDA calls its program of on-site inspections for GCP and GLP its “Bioresearch Monitoring Program” or “BIMO” • The program includes inspections of: – Clinical Investigators – Sponsors, monitors, CROs – Institutional Review Boards – Bioequivalence Laboratories and Facilities – GLP Facilities (nonclinical studies) • Each Center has its own BIMO group Woollen, 2002

CDER’s Division of Scientific Investigations Our BIMO People

DSI’s Program Responsibilities • Good Laboratory Practices • In vivo Bioequivalence • Good Clinical Practices – Institutional Review Boards (IRBs) – Clinical Investigators – Sponsor-Monitors, CROs

Program Objectives • To verify the quality and integrity of bioresearch data • To protect the rights and welfare of human research subjects

GCP Inspections: Routine vs. Directed • Routine – Inspections assigned for NDA/PMA’s • Directed – Problems identified at IND/IDE stage – Complaints to FDA • • FDA, other Agencies Sponsors/monitors Institutions/IRB’s Subjects/Public

CDER-GCP Bioresearch Monitoring (BIMO) Program • Clinical Investigator Inspection Program • Sponsor/Monitor/CRO Inspection Program • IRB/RDRC These “on-site” inspection programs collectively allow the agency to determine: – Adherence to applicable FDA regulations – Validity of data from studies in support of pending marketing applications (Data Integrity) – Whether the rights and safety of subjects have been protected (Safety) J. Rhoads, 2005

CDER BIMO Inspections (FY 2004) CI = 351 BEQ= 136 IRB = 120 GLP = 68 S-M = 17 Total = 692 J. Rhoads, 2005

When a Submission is Received: • Review Division Invites DSI staff for filing meeting • Team effort in selecting sites for inspection based on “risk based approach” – Impact to review of the study • Site(s) influencing significance on outcome • Outliers, e. g. , site with large proportion of treatment responders • Drop-outs; Adverse events; Protocol violations; large N# – Impact to the clinical trial process • Volume of work performed by the Clinical Investigator • Past inspectional history • Inspection request assignment to the field offices (ORA) • PDUFA Time Clock! J. Rhoads, 2005

Criteria for Assigning International Inspections International sites may be audited • if there are insufficient domestic data; • only foreign data are submitted to support an application; • domestic and foreign data show conflicting results pertinent to decision-making; or • there is a serious issue to resolve, e. g. , suspicion of fraud, scientific misconduct, significant human subject protection violations. J. Rhoads, 2005

After an Inspection • Form FDA 483: Inspectional Observations – Left with CI at close of inspection – Immediately available via FOI • Establishment Inspection Report (EIR) – Prepared by field investigator after inspection – Includes exhibits supporting observed deficiencies J. Rhoads, 2005

After an Inspection • Clinical Inspection Summary to the Review Division – Summary of all inspections assigned for an application – Recommend accept or reject data – Provided to review division in advance of PDUFA action goal date • Letter to the inspected party – emphasize deviation from regulations, if any. – Copied to the review division – include “DSI note to Review Div. Medical Officer” J. Rhoads, 2005

Regulatory/Administrative Followup • Rejection of study • Disqualification • Prosecution J. Rhoads, 2005

Clinical Investigator Deficiencies CDER Inspections - FY 2004 55% 48% 34% 28% 29% 21% 17% 7% 2% 5% 10% Foreign n = 71* Domestic n = 243* 4% *% does not include Biologic inspections J. Rhoads, 2005

Some GCP Challenges and Initiatives • Increased international inspections • Risk-based approach to inspection site selection • Accountability of study staff/site management organizations in clinical research • Linked real-time inspections (CI/Sponsor/IRB) J. Rhoads, 2005

Data Integrity/Quality: A Review Perspective • Medical Reviewers • Statistical Reviewers

NDA/BLA Review: Medical • • • Exclusion of patients from primary analyses Reliance on unplanned subset analyses Lack of consistency of results within and across studies Patient entry errors (cancelled patients, ineligible patients) Non-evaluable patients Missing data Inconsistent or clearly inaccurate data Extensive or non-random corrections to case report forms Missing source documents Possibility for bias in patient treatment or patient assessments … Robert De. Lap, Ph. RMA Meetings, Washington DC, October 26, 1994

NDA/BLA Review: Statistical • • Assess compliance with protocol / blinded analysis plans Assist DSI in planning investigator audits Check appropriateness of statistical models and conclusions. Verify results reported in the NDA. Modify models and assess robustness/sensitivity of the results. Modify data sets and reanalyze. Examine the trial and data for bias: • Results by center Baseline predictors Important subgroups (sex, age, race, etc, ) Assess impact of audits – – –

Case Study #1 “Unconscious Bias” and Readjudication The reduction in sudden deaths, said the FDA, had been exaggerated. Some patients who died according to Dr. Robert Temple Director of FDA’s Cardio-Renal Drug Products Division, had been excluded for “minor protocol violations”; and most of the excluded patients turned out to have been taking anturane … Nearly all the miss-classifications turned out to favor the hypothesis that anturane prevents sudden deaths. Lancet, August 9, 1980, p. 306

Case Study #1 “Unconscious Bias” and Readjudication The reason for these doubts is that Ciba Geigy, which had a stake in the outcome, not only paid the bills (around four million dollars) but also was deeply involved in the daily processing and collection of data before turning over the information to an independent policy committee. ” Lancet, August 9, 1980, p. 306

Case Study #2 Analyzing Potential Fraud Treatment-by-Center Interaction: p = 0. 056 2 Mean Change from Baseline 1 ? 0 -1 -2 -3 Drug X Placebo 12 7 10 5 19 9 1 7 19 0 4 1 2 3 4 5 CENTER 10 5 3 17 9 1 0 6 7 8 9

Case Study #2 Center 1 Center 8 Symp. C 0 1 2 3 0 68 4 0 0 1 17 4 0 0 2 16 21 2 0 3 1 0 0 0 Total: 133 0 Symp. B Analyzing Potential Fraud (con. ) 0 1 2 3 33 2 0 0 3 33 0 0 5 4 63 2 0 0 1 31 Total: 177

Case Study #2 Analyzing Potential Fraud Symptom Score Percent of Values 100 80 None 60 Mild 40 Moderate 20 Severe 0 1 2 3 4 5 6 7 8 9 Center

Case Study #3 Halcion: For-Cause Audit From the Summary Basis for Approval Protocol 6045 (investigators: Franklin, Howard, Reeves, Simson): 129 insomniacs (aged 22 -62 years) participated in four double-blind 28 -day studies comparing triazolam 0. 5 mg to placebo; Franklin and Reeves enrolled 78% of the patients. Pooled data showed triazolam was significantly better than placebo at end of week 1 on efficacy measures… Drowziness, dizziness, and impaired coordination were significant side effects; 13/61 patients in drug group who dropped out due to side effects, 10/13 dropped out in the first week; no abnormal changes in clinical significance were observed in lab analyses or vital signs.

Case Study #3 Halcion: For-Cause Audit Efficacy Analysis (Protocol 6045)

Case Study #3 Fraud: Robert Fiddes “If ever there was a wonder boy in the lucrative business of drug testing, it was Dr. Robert Fiddes. In just a few years, Fiddes transformed his sleepy medical practice. . . into a research juggernaut, recruiting his patients for drug experiments at a breakneck pace. His success made him a magnet for an industry desperately scouring the nation for test subjects. Companies large and small showered him not only with almost 200 studies to conduct, but with millions of dollars in compensation for his work. ” Kurt Eichenwald and Gina Kolata “A Doctor's Drug Studies Turn Into Fraud” New York Times, May 17, 1999

Case Study #3 Fraud: Robert Fiddes “The abuses of this one doctor point to weaknesses in the new system that has developed in recent years for testing experimental drugs. No longer does the pharmaceutical industry rely on career researchers at academic medical centers, whose professional reputations are forged on the quality of their data. Rather, the industry has turned to thousands of privatepractice doctors, for whom testing drugs has become a sideline for making money. ” Kurt Eichenwald and Gina Kolata “A Doctor's Drug Studies Turn Into Fraud” New York Times, May 17, 1999

When Things Go Wrong • Sensitivity Analysis (Reanalyze) • Expand Audit (FDA, Sponsor, Third Party) • Resubmission (delayed approval)

Changing Landscape • • The Critical Path Data standards & electronic submissions PRO and e. PRO Organizational Changes – Bimo Initiative – CDER’s new Office of Translational Science (OTS) • Connections

Critical Path http: //www. fda. gov/oc/initiatives/criticalpath/

Opportunities List

The Critical Path— Data Standards

FDA Vision: e-Review FDA Sponsor Gateway Sponsor Document Mgmt System Sponsor Data Warehouse e. Sub FDA Electronic Document Room Servers CRTs Repository Documents Document CRTs Share Listings Patient Profiles Analysis Data Web. SDM Data Load & Validation Janus Oracle Database Warehouse Review Environment Desktop Tools (Acrobat, PPV, JMP, Excel etc. ) Web. SDM Data Viewer

“The Standards Business” Kush/CDISC (2005) World Health Organization (WHO) International Conference on Harmonization (ICH) EFPIA JPMA EMEA MHLW Ph. RMA U. S. Dept. of Health and Human Services (HHS) U. S. FDA NIH/NCI CDC NLM RCRIM Technical Committee CDISC ADa. M SDS ODM LAB Med. DRA Protocol Representation e. CTD = Organization Health Level 7 (HL 7) Reference Information Model RIM SNOMED LOINC CDA BRIDG Model = Dictionary, Codelist = Standard = Model = Document Standard, or Architecture

FDA/CDISC www. cdisc. org • Clinical Data Interchange Standards Consortium • Workgroups – Operational Data Model (ODM) – Submission Data Standards (SDS) – Analysis Data Models (ADa. M) – Protocol – Electronic Source Data Interchange (e. SDI) – Lab Data

ODM: Clinical Data Structure *Subject. Data~ ? Audit. Record *Study. Event. Data~ ? Signature ? Audit. Record *Form. Data~ ? Investigator. Ref~ ? Signature ? Audit. Record *Item. Group. Data~ ? Site. Ref~ *Study. Event. Data~ *Annotation ? Signature *Annotation *Subject. Data~ *Annotation ? Archive. Layout. Ref~ ? Signature *Annotation ? Audit. Record *Item. Group. Data~ *Item. Data~ Kubick

ADa. M: Analysis-level Metadata • ANALYSIS NAME – A unique identifier for this analysis. May include a table number or other sponsor-specific reference. • DOCUMENTATION – A text description documenting the analysis performed. • REASON – The reason for performing this analysis. Examples may include Pre-specified, Data-driven, Exploratory, and Regulatory Request. • DATASET – the name of the analysis dataset used should be linked to the analysis dataset used for this analysis. In most cases, this will be a single dataset. If multiple datasets are used, they should all be listed here. • PROGRAM – Analysis programs using the DATASET above as input can be described or included here.

New Draft Guidance http: //www. fda. gov/cder/guidance/5460 dft. pdf

e. PRO: Specific Concerns When Using Electronic PRO Instruments • …sponsors should plan carefully to ensure that FDA regulatory requirements are met for sponsor and investigator record keeping, maintenance, and access. • These responsibilities are independent of the method … apply to electronic PRO data. • Sponsors are responsible for providing investigators with the information they need to conduct the investigation properly, for monitoring the investigation, for ensuring that the investigation is conducted in accordance with the investigational plan, and for permitting the FDA to access, copy, and verify records and reports relating to the investigation…

BIMO Initiative • Ref. “Modernizing Human Subject Protection/Bioresearch Monitoring, ” Rachel E. Behrman, Deputy Director, Office of Medical Policy & Lead, Cross-Center Initiatives Task Force presented at Ph. RMA Bimo Meeting, March 31, 2006

FDA’s HSP/Bi. MO Initiative • Part of FDA’s Critical Path Initiative • Begun December 2004 • Steering committee chartered – includes representatives from all centers and relevant offices • Scoped out dimensions of issues and formed working groups R. Behrman, 2006

FDA’s Oversight Must Evolve • Must provide regulatory guidance and perhaps new regulatory scheme that encompasses modern trial arrangements – Responsibilities of investigators – Data integrity • Must facilitate effective IRB oversight of evolving clinical trials arena to facilitate – IRB oversight of human subject protection – FDA oversight of IRB function • Need common standards and regulatory requirements for electronic data handling • Must be able to accommodate globalization of clinical trials • Must ensure comprehensive approach to protection of vulnerable populations R. Behrman, 2006

Guiding Principles of FDA Initiative • Collaborative efforts among government, academia, industry and patient groups • Infrastructure and “toolkit” development, not product development • Build support for academic science bases in relevant disciplines • Build opportunities to share existing knowledge & database • Develop enabling standards R. Behrman, 2006

CDER/OTS • CDER’s new Office of Translational Science • “Coming together” – Office of Biostatistics (OB) – Office of Clinical Pharmacology (OCP) • Focus for Critical Path activities at CDER

Comments/Connections • Parsimony: Guidance for Industry --Cancer Drug and Biological Products —Clinical Data in Marketing Applications • Critical Path: Case report form Standards • DQRI – Data Quality Research Institute • SCDM – Society for Clinical Data Management

Regulation Guidance The Terminator Mr. Rogers

Thank You stephen. wilson@fda. hhs. gov