Research Impropriety Alan K Jacobson MD ACOSRD Loma

Research Impropriety Alan K. Jacobson, MD ACOS/R&D Loma Linda VAMC

What have we covered so far? • Responsible • So what more could one ask for? • • • Institutions IRB Committees Investigators Auditing and Monitoring

Murphy’s Law If anything can go wrong, it will.

Major John P. Stapp, M. D. • • • Not well known in the compliance literature Human Guinea Pig, 1949 632 mph, 46. 2 Gs force “our good safety record was due to a firm belief in Murphy’s Law and in the necessity to try and circumvent it. ” Died age 89 of natural causes

So who was Murphy? • • • Captain Edward A. Murphy Edwards AFB Engineer on project MX 981 - deceleration When technician wired a transducer incorrectly - “If there is any way to do it wrong, he’ll find it. ” Recorded by George E. Nichols, Northrop project manager, who kept a list of laws.

Colonel John P. Stapp, M. D. “… a firm belief in Murphy’s Law and in the necessity to try and circumvent it. ” The Essence of Local Accountability

Research Impropriety • • • Why does it occur? What is it? How should it be managed? Can it be prevented? Is there any good news?

Research Impropriety “Does medicine have a culture that turns a blind eye to research misconduct? ” Dr. Richard Smith Former Editor, British Medical Journal

Research Impropriety: Why does it occur? • • • Why wouldn’t it happen? It happens in all other human activities. Pressure to publish. Inadequate training. Not taught good practice. Indeed, sometimes taught the opposite. Does sloppy behaviour spill over to fraud? You can get away with it. The system works on trust. Dr. Richard Smith Former Editor, British Medical Journal

Research Impropriety: What is it? Multitude of Terms • Research or Scientific • • • Impropriety Misconduct Misbehavior Noncompliance Fraud Sloppy science / research Junk science / research “Egregious abrogation of investigator responsibilities” “Scientists behaving badly”

“Scientists Behaving Badly” • • • Research on Research Integrity (Office of Research Integrity/NIH) Brian C. Martinson and colleagues 3, 600 surveys to NIH funded scientists 1, 768 (52%) usable responses % of scientists who say that they engaged in the behaviour listed within the past three years “… researchers can no longer afford to ignore a wide range of questionable behaviour that threatens the integrity of science. ”

Top Ten Scientific Misbehaviors: Likely or Very Likely to be Sanctionable 1. Falsifying or ‘cooking’ research data 0. 3% 2. Ignoring major aspects of human subject requirements 0. 3% 3. Not properly disclosing involvement in firms whose products are based on one’s research 0. 3% 4. Relationships with students, research subjects or clients that may be interpreted as questionable 1. 4% 5. Using another’s ideas without permission or due credit 1. 4% 6. Unauthorized use of confidential information in connection with one’s own research 1. 7% 7. Failing to present data contradicting one’s own previous research 6. 0% 8. Circumventing minor aspects of human subject requirements 7. 6% 9. Overlooking others’ flawed data or questionable interpretation of data 12. 5% Changing the design, methodology or results of a study in response to pressure from 10. a funding source Martinson et al, “Scientists Behaving Badly”, Nature 2005; 420: 739 -740 15. 5%

Scientists Behaving Badly: Other Behaviours 11. Publishing same data or results in two or more publications 4. 7% 12. Inappropriately assigning authorship credit 10. 0% 13. Withholding details of methodology or results of a study in papers or proposals 10. 8% 14. Using inadequate or inappropriate research designs 13. 5% 15. Dropping observations or data points from analyses based on a gut feeling that they were inaccurate 15. 3% 16. Inadequate record keeping related to research projects 27. 5% Martinson et al, “Scientists Behaving Badly”, Nature, 2005

The Spectrum of Research Impropriety Administrivia administrivia - the tiresome but essential details that must be taken care of and tasks that must be performed in running an organization; "he sets policy and leaves all the administrivia to his assistant” The. Free. Dictionary Fatal / Criminal

Nothing New Under the Sun • • • Charles Babbage 1791 - 1891 Primary Legacy - Inventor of “Difference Engine” - first advance in computers since abacus Less renown - author on dishonesty in science • Trimming - “clipping off little bits here and there” • Cooking - selective reporting of a group of results • Forgery - made up results

History of Research Impropriety: Failure of an entire society

History of Research Impropriety: Failure of Oversight Committees 1931 Rosenwald Fund cuts support to development projects. Clark and Vondelehr decide to follow men left untreated due to lack of funds in order to show need for treatment program. 1968 Concern raised about ethics of study by Peter Buxtun and others. 1969 CDC reaffirms need for study and gains local medical societies' support (AMA and NMA chapters officially support continuation of study). 1972 First news articles condemn studies. 1972 Study ends and participants compensated with cash and continued medical treatment. 1973 Congress holds hearings and law suit initiated. 1997 On May 16 th President Clinton apologizes on behalf of the Nation.

History of Research Impropriety: Failure of Investigator Responsibilities • Dr. William Summerlin, Oncology (1974) • • Dr. John Darsee, Cardiology (1981) • • • Left research, barred from federal funds Brigham & Women’s had to return NIH funds Eric Poehlman, Aging, (2000 -2005) • • • Never returned to research Fined $180, 000; barred from federal funds Currently in federal prison Hwang Woo Suk, Stem Cell Research (2005 -2006) • Fired from Seoul University, possible criminal charges

Research Impropriety: Why Should WE Care? • The Hall Affair • “Sadly, as it unfolded, the Hall affair illustrated the reality that allegations of research impropriety affect the careers of both the accused and the accusers and, in the process, can divide an institution and damage its reputation” M. B. Van Der Weyden http: //www. mja. com. au/public/issues/180_04_160204/van 10035_fm. html • Imanishi-Kari / Baltimore / O’Toole • • Post-doc accuser 1986, cleared by MIT, Tufts & NIH investigations, but congressional panel and secret service investigations resulted in 1994 ORI ruling of 10 years barred from federal funds 1997 DHHS arbitration panel overturned ORI ruling

The Cost of Research Impropriety I • Research Integrity Cost • • • Scientific Record Public Confidence “Impropriety of any type in the conduct of research is abhorrent to the inherent purpose of all scientific inquiry: the discovery and dissemination of truth. ” Research Related Policies and Procedures State University of New York (SUNY)

The Cost of Research Impropriety II • Cost to subjects and patients • President Lincoln’s commitment: • “…to care for him who shall have borne the battle, and for his widow and his orphan. ”

The Cost of Research Impropriety III • • Direct Investigative and Management Cost Distraction Cost • What isn’t getting done while resources are utilized to investigate and manage the impropriety and it’s fallout.

Impact of Research Impropriety

Research Impropriety So, What IS it?

Definition of Research Impropriety Under VHA Handbook 1058. 2 “Research impropriety is any ethical lapse or other impropriety involving or occurring in connection with research other than research misconduct…” as defined in the Federal Policy on Research Misconduct VHA Handbook 1058. 2 “Research Misconduct”

Types of Research Impropriety Research Misconduct “All Other” Research Impropriety

Types of Research Impropriety Research Misconduct “All Other” Research Impropriety

Research Misconduct: VHA Policy Conforms to Federal Policy • Research misconduct is defined as fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results. • Research misconduct does not include honest error or differences of opinion. Federal Register Notice Vol. 65, No. 235, Dec. 6, 2000

Research Misconduct: Definitions • Fabrication (1058. 2 - 5 c) • • Falsification (1058. 2 - 5 d) • • Making up data or results and recording or reporting them Manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record Plagiarism (1058. 2 - 5 i) • Appropriation of another person’s ideas, processes, results, or words without giving appropriate credit

Research Misconduct: Definitions • Research Integrity Officer (1058. 2 - 5 l) • • • The RIO is the appointed official at each VA facility who is responsible for receiving and coordinating reviews of formal allegations of research misconduct. Role further defined in 1058. 2 - 7 Liaison with ORO 1058. 2 - 7 b(1)

Responsibilities of Federal Agencies and Research Institutions • The Federal Policy on Research Misconduct applies to “all federally-funded research and proposals submitted to Federal agencies for research funding. ” • “Federal agencies have ultimate oversight authority for Federally funded research, but research institutions bear primary responsibility for prevention and detection of research misconduct and for the inquiry, investigation, and adjudication of research misconduct alleged to have occurred in association with their own institution. ” Federal Register Notice Vol. 65, No. 235, Dec. 6, 2000

Research Misconduct: Definitions II • • • Federal and VHA definitions are extremely precise Research Misconduct is specifically restricted to FFP Non-federal institutions, i. e. affiliated universities, may have variable policy definitions, even though they are required to follow federal policy on federally funded projects

Research Misconduct: How Should Allegations be Handled? • VA Handbook 1058. 2 identifies individuals responsible for investigating allegations of misconduct and provides guidance for procedures and roles. • Responsible authorities must determine whether the conduct was deliberate, or honest error. • Investigations must be conducted confidentially and with discretion to protect both informant and accused parties.

Research Misconduct: Management • VERY detailed process • In summary, if a question of Research Impropriety arises and there is any hint that it may be related to FFP - DO NOT START LOCAL INVESTIGATION ! • If any question of FFP - notify RIO and have RIO coordinate with Peter Poon in ORO • The evaluation will likely be done locally, but due to the complexity of the process it should be coordinated with ORO.

Research Misconduct: Management • Defined Phases • • • Allegation Inquiry Investigation Adjudication Departmental Review Appeals

Research Misconduct: Corrective Actions - Considerations • • • The extent of the misconduct (amount, duration, scope) Degree to which the misconduct was knowing, intentional or reckless Presence or absence of a pattern of misconduct The consequences or possible consequences of the misconduct The respondent’s position and responsibility for the project Cooperation of the respondent during Inquiry and Investigation Likelihood of rehabilitation Presence of similar cases - commensurate action Any other extenuating or aggravating circumstances

Research Misconduct: Examples of Corrective Actions • • • Government-wide debarment (in development) Removal from a particular project, suspend/terminate award Restitution of funds or civil penalties Prohibition from receiving VA research funds for a period Correction or retraction of published article Monitoring or supervision of future work Required certification of data Required certification of sources (references and contributors) Remedial education or mentoring

Types of Research Impropriety Research Misconduct “All Other” Research Impropriety

“All Other” Research Impropriety • “All Other” Research Impropriety is as ill-defined as Research Misconduct is rigorously defined

The Spectrum of Research Impropriety Administrivia administrivia - the tiresome but essential details that must be taken care of and tasks that must be performed in running an organization; "he sets policy and leaves all the administrivia to his assistant” The. Free. Dictionary Fatal / Criminal

“All Other” Research Impropriety: Definitions / Examples “Examples of research impropriety include, but are not limited to, conflicts of interest, misallocation of funds, sexual harassment, discrimination, and breaches of human subjects protections and animal welfare requirements. ” VHA Handbook 1058. 2 “Research Misconduct”

“All Other” Research Impropriety: Examples • Failing to retain significant research data for a reasonable period; • Maintaining inadequate research records, especially for results that are published or are relied on by others; • Conferring or requesting authorship on the basis of a specialized service or contribution that is not significantly related to the research reported in the paper; • Refusing to give peers reasonable access to unique research materials or data that support published papers; Responsible Science: Ensuring the Integrity of the Research Process, National Academy of Sciences/Institute of Medicine Report, 1992

“All Other” Research Impropriety: Examples • Using inappropriate statistical or other methods of measurement to enhance the significance of research findings; • Inadequately supervising research subordinates or exploiting them; and • Misrepresenting speculations as fact or releasing preliminary data to allow peers to judge the validity of the results or to reproduce the experiments. Responsible Science: Ensuring the Integrity of the Research Process, National Academy of Sciences/Institute of Medicine Report, 1992

“All Other” Research Impropriety: Examples • Engaging in inappropriate authorship practices on a publication and failing to acknowledge that data used in a grant application were developed by another scientist • Inappropriate data analysis and use of faulty statistical methodology • Failure to document and incorporate subject’s feedback in findings • Misuse or misallocation of funds for unauthorized purposes • Financial conflict of interest of self or close relatives (e. g. , spouse) • Time and effort reporting errors or omissions or over-commitment • Inappropriate staff assignments • Neglect of research-related administrative tasks

“All Other” Research Impropriety: Compromise of Research Subject Protections • • Inadequate adverse event reporting Privacy violations Breach of patient confidentiality Failure to inform subjects of new information Failure to warn subjects of possible side effects Failure to ensure that subjects understand informed consent Failure to conduct adequate literature review prior to starting clinical trials

“All-Other” Research Impropriety • Authorship (other than plagiarism which is research misconduct) • Conflict-of-interest • Noncompliance • Oversight / Supervision • Sloppy Science

Adherence to Editorial Guidelines • International Committee of Medical Journal Editors (ICMJE) guidelines for: - Authorship and Contributorship - Peer Review - Conflicts of interest - Privacy and Confidentiality - Protection of Human Subjects - Corrections, Retractions, Expressions of “Concern”

Conflict of Interest

“All Other” Research Impropriety Management • • • Research Issue or Personnel Issue No proscribed initial investigatory process What to report to ORO • • • September 2005 Guidance Document VHA Handbook 1058. 1 VHA Handbook 1058. 2

ORO Reporting: September 2005 Guidance Document • What to Report to ORO: • • Protection of Human Subjects in VA Research Laboratory Animal Welfare in VA Research Safety and Security in VA Research Misconduct in VA Research

ORO Reporting - Human Subjects I • • • Any adverse event in a human subject, or an imminent threat of an adverse event, that results in a substantive action by the IRB under 1058. 1 Any unexpected death of a human subject under 1058. 1 Any unanticipated problem involving risks to subjects or others that results in a substantive action by the IRB Any for-cause suspension or termination of VA human subject research by the IRB, the VA facility, or a VA affiliate institution Any serious or continuing noncompliance with federal regulations or VHA policies for the protection of human subjects, including: • 38 CFR Part 16 • 45 CFR Part 46 • 21 CFR Parts 50, 56, 312 or 812 • VHA Handbook 1200. 5

ORO Reporting: Human Subjects II • Any serious or continuing noncompliance with federal regulations or VHA policies for the protection of human subjects, including: • 45 CFR Part 46 (Protection of Human Subjects in Federal Research) • Based on Belmont Report, became law in 1981 • Subpart A - Basic HHS Policy for Protection of Human Research Subjects • Subpart B - Additional protections - pregnant women, fetuses, neonates • Subpart C - Protections for Prisoners • Subpart D - Protections for Children • 38 CFR Part 16 (Protection of Human Subjects in VA Research) 1991 • VA promulgation of The Common Rule, identical to subpart A, 45 CFR 46 • FDA 21 CFR • Part 50 (common rule) • Part 56 (IRBs), • Part 312312 (IND) • 812 (IDE) • VHA Handbook 1200. 5 • Requirements for the Protection of Human Subjects in Research

ORO Reporting - Human Subjects III • • • Any serious or continuing noncompliance with IRB requirements or determinations Any findings of noncompliance in human research protections from ORD or other VA office Any findings of noncompliance in human subject protections from external oversight agencies such as OHRP, FDA, Any change in the facility’s accreditation status from a VArecognized accreditation organization for human research protections Any change in the facility’s FWA or designated IRB’s as filed with OHRP Any significant change in the facility’s MOU with an affiliate institution or other VA facility regarding the designation of IRB(s) or other human research protection function

“All Other” Research Impropriety Management • • Research Issue or Personnel Issue If it is a Personnel Issue: • • Is it a conduct issue? Is it a performance issue?

“All Other” Research Impropriety Corrective Action / Consequences • Examples from Research Misconduct • • • The extent of the misconduct (amount, duration, scope) Degree to which the misconduct was knowing, intentional or reckless Presence or absence of a pattern of misconduct The consequences or possible consequences of the misconduct The respondent’s position and responsibility for the project Cooperation of the respondent during Inquiry and Investigation Likelihood of rehabilitation Presence of similar cases - commensurate action Any other extenuating or aggravating circumstances

Research Impropriety: Management When Scientists Do Bad Things • • Does it involve FFP? Is it reportable to ORO? Is it a Personnel Issue? Otherwise …

Research Impropriety: Can it be Prevented? • Educate - Training • • • GCP, HRP, HIPAA, Privacy Verify - Monitoring and Audits • Routine • For-cause Provide a compliant environment • • Accreditation • AAHRPP Exemplary Leadership Culture of Responsible Research Culture of Compliance

Research Impropriety: Can it be Prevented? • • Organizational Options - VHA Institutional Options - individual facilities Division / Research Group Options Individual Investigator Options

Finding the Balance • Effective assurance of “responsible conduct of research” • Suffocating oversight and bureaucracy

Research Impropriety: Can it be Prevented?

Research Impropriety: Can it be Prevented? • Steinberg Tips: • • • Make your position clear - Impropriety will not be tolerated and will be caught Avoid hyperbole and jokes about getting the data no matter what Don’t “improve” the current study - let staff know that the current protocol is the only one approved Promote Research Integrity Verify, Verify

Research Impropriety: Can it be Prevented? ORI Introduction to Responsible Conduct of Research Nicholas Steneck, Ph. D This booklet introduces the reader to the nine RCR core instructional areas in four sections that follows research from inception to planning, conducting, reporting, and reviewing research. The publication features case studies, textbox inserts, discussion questions, and electronic and printed resources.

Colonel John P. Stapp, M. D. “… a firm belief in Murphy’s Law and in the necessity to try and circumvent it. ” The Essence of Local Accountability

Research Impropriety: The Good News • • Documented Research Impropriety is relatively rare ORI statistics

Useful Resources VA Office of Research & Development (ORD) Program for Research Development & Education (PRIDE) http: //www 1. va. gov/resdev/programs/pride/ K. Lynn Cates, MD, (202) 254 -0282, lynn. cates@va. gov Marisue Cody, Ph. D, (202) 254 -0271, marisue. cody@va. gov VA Office of Research Oversight (ORO) http: //www 1. va. gov/oro/ Research Misconduct: Peter Poon, JD, MA (202) 565 -8107, peter. poon@va. gov HHS Office of Research Integrity (ORI) http: //ori. hhs. gov/ HHS Office for Human Research Protection (OHRP) www. hhs. gov/ohrp