Human Research Protection Program and QI of TMUH

Human Research Protection Program and QI of TMUH Sung Hui Tseng MD Ph. D 2019 -6 -18

Objectives : HRPP organization structure, policies and procedures of TMUH Collecting Data to measure safety, efficiency, effectiveness High Risk Areas identification About Quality Improving Plan 2

AAHRPP Standards and Philosophy • Self-assessment and peer review • Raises participant protection to an organizational priority • More visible role of IRB/EC • Better efficiency and streamlining • Improved communication/operational transparency • Attract and retain high level researchers • Increases ability to act nimbly and responsively to ever-changing research landscape 3

AAHRPP精神與人體研究保護方案(HRPP) Share responsibility throughout the organization 2016 AAHRPP Conference, At the Crossroads: Harmonizing Ethics, 4 Efficiency, and Innovation

再認證主責單位 Domain II Domain I 條文: Standard (8) Organization Institutional Review Board 條文: Standard (5) Researcher and Research Staff Domain III 條文: Standard (2) 6

Flowchart of A Typical Research Project Post Publication Audit TMUH-DMR PI: Design research Protocol review / COI deliberation Publication Approval HRPP Audit Conduct the research Monitor Research site approval Contract, Affidavit TMU-ICIOC TMU-JIRB TMUH-DMR

3 Key Elements Consolidate HRPP Performance Researchers and DMR Research Staffs TMU-JIRB Compliance DMR Oversight DSMS Department of Preventive and Community Outreach Community Medicine Education CHRP Program TMU-JIRB Conflict of Interest (COI) Organizational TMUH Plan Department Pharmacy of Services Pharmacy Contracts ICIOC and Grants IRB TMU-JIRB 8

TMUH HRPP Organization Structure and Leadership Chair: Superintendent Director: Vice Superintendent of Research

Our Goal: Re-Accreditation

全面稽核與發現 感謝~ Top-Down Engagement

Accomplishing our Goal: Accreditation

Quality Improvement (QI) • “The systematic approach to reduction or elimination of waste, rework, and losses in production process. ” • “To identify opportunity for process improvement, address potential problems and promote best practice. ” • To correct workflow processes, improve efficiencies, reduce variations in outputs, and address areas of non-compliance. • A Performance enhancement in response to observed trends or findings • QI can take many forms, but the philosophy remains: – The systematic and continuous actions that lead to measurable improved performance. 18

Data and Quality Improvement • Every HRPP wants to improve its processes, its compliance, its efficiency. • Within every HRPP lies a mountain of ‘data” either already collected or collectible – Data includes • Audit and monitor reports • Research team reports to HRPP: noncompliance, deviations, adverse events, etc. • Data collected by the HRPP • From its records • From specific purposeful outreach efforts 19

What Do We QI • • • Regulatory Compliance Community Satisfaction/Dissatisfaction Data Quality Efficiency Each Involves Assessments of – – – Policies Procedures/work processes Operations

Quality Improvement Plans • • To Err is human: the major cause of adverse events is poorly designed system. Quality improvement activities are an important component of organization operations. • QI activities are systematic, data-driven actitivies and involve human participant, when is IRB approval needed ? Research ethics committees in the regulation of clinical research: comparison of Finland to England, Canada, and the United States (Hemminki E 1, 2016) – Conclusion: In England, quality assurance of REC work was thorough, fairly thorough in the USA, and not performed in Finland. The work of RECs in the four countries varied notably. Various deficiencies in the system require action, for which international comparison can provide useful insights. Quantify HRPP quality – Creating investigator profile

Performance evaluation: measurement of safety, efficiency, effectiveness • • • Information technology User survey Audit findings analysis JIRB and institution review turnaround time SAE analysis 22

QI: Compliance • Analysis of Data from Audit Checklist • Improvement of efficiency and effectiveness of research compliance program – Goal: Using minimum data sets as Compliance Quality Indicators to improve outcomes and processes and patient safety 23

Risk based audit 1. 2. 3. 4. 5. 6. 7. High-risk human trial/study Having serious unanticipated problems that may harm participants or others. Participants are vulnerable populations: children, prisoners, minors, pregnant women, handicapped or mentally disabled persons, students, subordinates (affiliate relationship), terminal patients, and so on. Trials and researches initiated by investigators. Having serious adverse event reports Abnormal events or unexpected events occur, such as the appeal cases, exposed by the media, or other non - compliance violations or special cases. Others if needed.

Incidence rates of noncompliance General back ground • Not-for-cause audit in 2016 was 45. 5% and 36. 2% in 2018. • Risk based audit: 70% What are the patterns of noncompliance in different types of research or researchers ? • Investigator initiated trials vs sponsor trials 25

Comparison of audit findings of Investigator Initiated Clinical Research to risk based audit AE/ SUSAR/ NC/ UP/ DSMB report 14% ICF related 29% IRB related 14% Delegation and Management 14% Data of subject 29% Investigators initiated research have higher incidence rates of unreported AE/SAE/NC/UP? DSMB reports Why? • • People don’t know what they don’t know even experts are often clueless as to what they don’t know.

Comparison of audit findings of Investigator Initiated Clinical Research to sponsor initiated trials N % PI initiated 323 76. 2% sponsor 101 23. 8% finding rate 2017 2018 2019 sponsor 66. 7% 22. 2% NA PI initiated 77. 8% 58. 3% # of protocols audited # of 2017 2018 2019 amendment sponsor 53. 8% 52. 1% 47. 1% PI initiated 46. 2% 47. 9% 52. 9% PI initiated researches have higher finding rates of deficiencies, less self reported NC/ UAP, and submitted less amendment reviews. "The devil is in the detail“ 36 25 12 Self reporting　 2017 2018 2019 # of NC/UAP reports NC UAP sponsor 88 11 97 8 29 2 PI initiated 8 1 17 2 9 2 subtotal 96 12 114 10 38 4 total 108 124 42 Post audit reporting　 NC/UAP reports after audit sponsor PI initiated subtotal 2017 NC UAP 0 0 1 0 1 2018 NC UAP 2 1 5 0 7 1 8 2019 NC UAP 0 0 1 1 2 27

Use minimum data sets as Compliance Quality Indicators to improve outcomes and processes and patient safety • • Focus Audit on patient-centered metrics and researchers at risk of jeopardizing the rights and welfare of subjects – Investigators initiated trials Use areas of higher deficiency rates to guide future oversight activities and education Facilities at risk, while constituting 25% of all facilities, contributed 70% ± 25% (M ± SD; range = 32%-100%) of all reported noncompliance/incidents in the following 14 performance metrics : Missing ICDs/Incorrect ICDs used/ICDs not signed by subjects/HIPAA authorization not obtained/Local adverse events determined to be serious, unanticipated, and related to research/Resulted in hospitalization/Resulted in death/Lapsed in IRB continuing reviews/ Suspended due to human subject concerns/Suspended due to investigator-related concerns/ Informed consent not obtained prior to initiation of study/Subjects included in research not meeting inclusion/exclusion criteria/Without initial training/Lapse in continuing training (Tsan MF & Nguyen Y, 2018)

Quality Improvement of Research Oversight with Electronic Audit System IRB continuing review

30

New Protocols Reviewed by TMUJIRB No. 1000 931 900 800 730 792 843 641 700 604 600 500 423 474 400 300 200 168 100 0 2011 2012 2013 2014 Academic Year(Aug-Jul) 2015 2016 2017 2018(~Mar) 31

IRB review efficiency analysis 45 • 40 35 30 25 20 IRB started on-site consulting service in Feb 2017. The IRB staffs provide face to face meeting at TMUH. Consulting issues: – Reviewer’s opinion – How to use the IRB submission system. IRB on-site consulting started Web based IRB submission started 15 • 10 5 Jan-Mar Apr-Jun Jul-Sep Oct-Dec Jan-Mar Apr-Jun Jul-Sep Oct-Dec Jan-Mar Apr-Jun 0 2013 2014 2015 2016 average PI response days Average Reviewer Review days Average Administration days 2017 2018

Satisfaction/Dissatisfaction • Satisfaction/Dissatisfaction monitoring plan – Annual User Surveys on HRPP components – Annual survey of researchers and staff awareness of HRPP – Auditing Satisfaction Survey

We believe… • • • Informed consent is central to the protection of human subjects (Byerly WG, 2009) IRB review is integral to ensure regulatory compliance and ethical conduct of research involving human subjects (Byerly WG, 2009) IRB letters are central to the practice of research ethics (Clapp JT, 2017) Auditing and accreditation programs can improve the quality of ethics review (Coleman CH et al 2008) Review by research ethics committees is the key in medical research regulation (Hemminki E, 2015)

Future directions Actual impact on research practice? (Coleman CH et al 2008) • Does IRB improves participants understanding risk and potential benefit? • Is IRB guidance actually been followed? • Is research risk reduced? • Does it change subject’s experience or attitude about research? • Does it result in more research responsive to local community’s self-identified needs? 35

Publications about HRPP • Minimal risk projects should entail minimal oversight (Fiscella K, 2015) • Unaware practitioners become involved in low-risk research activities without knowing it (Pediatrics 2017) High Low Regulation complexity Confidence of IRB members in their assessment of human research risk (Grinnell F, 2017) As risk increased, confidence decreased (Grinnell F, 2017) Min risk High risk 36

THANK YOU ! 39