Research Integrity Oversight A view from within the

Research Integrity Oversight: A view from within the system Pedro L. Muíño, Ph. D. Research Integrity Officer Saint Francis University Ethics Forum June 24, 2011

Research Misconduct Fabrication, Falsification, Plagiarism Source: Office of the Inspector General-National Science Foundation (2007)

Cheating and Research Misconduct (RM) RM means fabrication, falsification, or plagiarism in proposing or performing research [], reviewing research proposals [] or in reporting research funded by [the agency]. 45 C. F. R. 689. 1. a Fabrication: Falsification: making up data or manipulating results and materials, equipment, recording or or processes, or reporting them changing or omitting data or results overhead from NSF-OIG outreach materials (modified) Plagiarism: appropriation of another person’s ideas, processes, results or words without giving appropriate credit.

Let’s start with a case study ► A “fabrication” (? ) of credentials case ► Resource: § On Being a Scientist. Responsible Conduct in Research, 2 nd edition, page 17. Committee on Science, Engineering, and Public Policy. National Academy Press. Washington, D. C. 1995.

Questions Do you agree with Don that scientists often exaggerate the publication status of their work in written materials? 2. Do you think the department acted too harshly in dismissing Don from the graduate program? 3. Do you believe that being in “good standing” should be a prerequisite for obtaining an advanced degree in science? 4. If Don later applied to a graduate program at another institution, does that institution have the right to know what happened? 1.

Overview of the National Science Foundation and the Office of Inspector General Source materials for this and following slides: NSF-OIG outreach materials

Origins of NSF 1950: NSF Act Signed by President Truman “To promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…“ “Encourage & develop a national policy for the promotion of basic research and education in the math, physical, medical, biological, engineering and other sciences”

NSF is the only federal agency whose mission includes support for all fields of fundamental science and engineering, except for medical sciences. Annual budget of about $7. 424 B (2011). Funding ~20% of all federally supported basic research conducted by US colleges and universities. Cooperative efforts with many federal agencies (Do. D, Education, State, Do. E, HHS, NEH, Smithsonian. . . ) Nano. Wire electron paths in a nanowire, including imperfections in the wire. Credit: Eric J. Heller, Harvard University

NSF Support for Basic Research at Academic Institutions* * as of 2008

External Funding!! ► About 95% of NSF funds go outside the agency for grants to support research ► About 5% of NSF funds supports NSF staff ► About 1, 300 staff annually manage 40, 000 proposals and 40, 000 awards (10, 000 annual awards x 4 year average administrative duration)

Responsible Professional Practices ► Educating professionals on behaving professionally ► Teaching integrity, not merely the technical “rules of the road” ► Ensuring the continuity of world class research that is: ► honorable ► trusted ► relied on ► Facilitating collaborations, common understandings of rules, expectations

Expectations ► Simultaneously and Consistently § Act with Integrity § Foster Integrity § Ensure Integrity § In every research activity (funded, unfunded, local, federal, international, commercial, academic)

Office of Inspector General (OIG) ► Almost every federal agency/entity has an IG ► An IG is an independent office for oversight § Promote economy, efficiency, and effectiveness… § Prevent and detect fraud, waste, and abuse… …in agency programs and operations ► NSF OIG § 38 audit staff, 22 investigative staff § Investigations staff includes: Ph. D. scientists CPAs Special agents Attorneys

What’s an OIG? G. Trudeau Doonesbury, 5 December 2005

NSF-Office of Inspector General ~ 60 OIG employees Inspector General Allison Lerner Deputy Inspector General Tim Cross Counsel to the Inspector General Associate IG for Audit Administrative Staff Audit Staff Associate IG for Investigations Investigative Staff Scientific Police Legal

The Office of Investigations Associate IG for Investigations Peggy Fischer, Ph. D. Administration (2) Scientific Investigators (7 Ph. D, JD) Special Agents (6 SA) Legal and Outreach (6 JD, Ph. D)

Common Types of Administrative Allegations Data gathered from NSF OIG closed Investigative files (1990 – Present)

Common Types of Civil/Criminal Allegations *Includes mail fraud, false identification insurance fraud, impersonating a government officer, and copyright infringement. Data gathered from NSF OIG closed Investigative files (1990 – Present)

Typical Civil/Criminal Cases ► Grant Fraud ► Travel fraud, purchase cards, cost sharing, time and effort, stolen equipment ► Contract Fraud ► Overcharging ► Institutional Fraud ► Cost sharing, commingling funds, credit cards, false certifications, summer salary ► SBIR (Small Business Innovation Research) Fraud ► Duplicate efforts, general theft (funds paying for houses, furniture, children’s education, …) ► Employee Fraud ► Time sheets, pornography, credit cards

Investigative Outcomes * Allegation was preliminarily investigated but found to be insufficiently material to warrant further action. ** Includes monitoring, enacting new guidelines, letters of reprimand, etc. Data gathered from NSF OIG closed Investigative files (1990 – present)

Consequences of Research Misconduct ► Debarment ► Recovery of funds ► Certifications ► Assurances ► Fraud cases Last of materials adapted from NSF-OIG outreach documentation

How does the process work? ► How does OIG know what to investigate? § Allegations (anonymous or in person) § Internal investigations of grants ► Triage: What to do with the allegation? § Depending on the nature of the alleged event ► Civil/Criminal vs Scientific § If civil/criminal ► Special Agents and Lawyers may take the lead ► Accountants may need to be involved (AIG for Audit) § If scientific, what to do depends also on the alleged act ► Conduct investigation to probe validity of allegations ► Refer to institution for internal investigation

A look at scientific investigations OIG evaluates the evidence and decides whether it appears to have merit or not ► If it does not, the case is closed ► If it does, an investigator initiates data collection ► § OIG has the power to request any document related to the research being funded § Most institutions comply with minimal interference OIG is fairly sure of the facts, it informs the institution of the problem and asks (the institution) to conduct a thorough internal investigation ► After the institution conducts the investigation, it must submit to OIG a detailed report of its findings and adjudication ►

Why this method? ► OIG’s first priority is to secure the evidence § By requesting the relevant materials to be submitted to OIG immediately, the investigators ensure that the subject does not destroy evidence ► Institution conducts internal investigation § This is more likely to protect the subject’s reputation. After all, it is likely the allegations are false or the result of a misunderstanding § It is less intrusive for an institution to evaluate the problem rather than have federal investigators on campus

Resolution? ► OIG receives the report § If OIG agrees with it, OIG uses the findings to proceed by: ► Closing the process if no substantial findings were made ► Contacting the subject to hear allegations § If OIG disagrees with it, it may conduct its own on-site investigation ► After OIG is satisfied and hears the subject’s allegations, it submits a report and action recommendations to NSF’s deputy director ► The deputy director contacts the subject and informs him/her of the penalties levied

What happens at the end of the inquiry/investigation? ► Case closed for lack of evidence Plurality of cases ► If sufficient evidence: § OIG reports to the decision maker: ►DOJ for Civil/Criminal ►NSF Office of the Director for RM and other regulatory issues (this is what we just addressed in the previous slides)

Possible Outcomes ► Whatever sanctions the institution makes From the NSF/Federal side of the issue. . . ► ► ► ► Letter of Reprimand Ban from serving as a reviewer Ethics Training Certifications Assurances Federal-wide Debarment Fines / Restitution Prison overhead from NSF-OIG outreach materials (modified) drawing by Terri Groat-Ellner

Lengthy cases… most of the time ► Most cases take a couple of years to evaluate from the time of allegation to the time of adjudication § They involve a team of investigators throughout the different stages of the process ► Some are simple enough that can be completed in a short period of time ► Let’s see one such case, for which I was the only investigator involved

The case of Jack Silverman (not his real name) Dr Silverman submitted a proposal to NSF in 2007 ► Early 2008, the program officer contacted OIG ► § One of the reviewers was very concerned because Dr Silverman had listed a publication in the proposal as “in press”, when he (the reviewer) knew for a fact that this was not the case ► I eventually contacted the reviewer § It turns out that he was the editor of the journal § We had thought the problem was that the manuscript was still under review § It turns out that the manuscript had been rejected before Dr Silverman had submitted the proposal ► However…

The case of Jack Silverman ► OIG is very concerned about protecting the confidentiality of its informants § In fact, it may choose to forego an investigation if following it may compromise confidentiality ► In this case, the reviewer was concerned that his name would eventually come out, because he was probably the only person who knew this paper had been rejected ► So, we looked for other potential problems among the listed papers

Pattern of exaggeration ► Of the other 9 papers, 5 had “omissions” § For example, what looks better: ► Silverman et al. Science News (2006), 350, 121 -122 ► or Silverman et al. Science (2006), 350, 121 -122? § Or: ► Smith, J; Silverman, J; Johnson, A. JACS (2007)… ► or Silverman, J. JACS (2007)…? ► Similarly for titles: § “Director of …” vs “Assistant Director of …” ► Or for conferences: § “Paper presented at the AAAS meeting” § vs “Paper presented at the AAAS %%%%% regional meeting”

Resolution ► I contacted Dr Silverman ► Eventually, the OIG office decided that this case was not serious enough to warrant any further action § I asked about each of the individual cases of exaggeration § Also noted to him that the same problems appeared on his vita, (posted online) § He said that he was a bad typist and the mistakes, once made, were perpetuated by coping and pasting § He eventually volunteered the information that the paper that prompted the investigation had been rejected (so the reviewer was off the hook) § It was considered to be “Questionable Research Practices”, not “Research Misconduct” ► A letter of warning was sent to Dr Silverman

http: //www. nsf. gov/oig/search/A 08010001. pdf

Note the redacted information in order to protect confidentiality Once a subject has completed his/her penalty, s/he is entitled to start anew (remember Don’s case study? ) http: //www. nsf. gov/oig/search/A 08010001. pdf

Cases studies Taking the crooked road… overhead from NSF-OIG outreach materials (modified)

Case Study #1 ► ► ► Subject: An assistant professor. The allegation: He knowingly falsified data for a series of experiments published in a journal article. University actions: Initially did not conduct a detailed investigation. When OIG showed intent to investigate on its own, the university reopened the case. How did he do it? He manually changed the settings on an instrument so the produced data matched his desired outcome. He then relied on his co-author to fit the curves because the co-author had better math skills. Significance of the actions: The actions were serious and frivolous. The scientific record was affected and the paper eventually retracted. Actions: The subject was debarred for a period of two years. For a period of three years after debarment, the university must certify that all proposals submitted by the subject do not contain plagiarized, fabricated, or falsified material. ► The subject must undertake ethics training ► ► overhead from NSF-OIG outreach materials (modified)

Case Study #2 Subject: A post-doctoral fellow. The allegation: He knowingly falsified data included in a grant proposal. Institutional actions: After admission of wrongdoing by the subject, he was fired. ► How did he do it? He included falsified data and inappropriate mathematical manipulations. He reported two experiments when only one was done. He falsified images by coloring cells. ► Significance of the actions: The actions were very serious, although the scientific record was not affected. ► Actions: ► ► The subject was debarred for a period of two years. ► He lost all financial assistance at his institution. ► The subject must undertake ethics training overhead from NSF-OIG outreach materials (modified)

overhead from NSF-OIG outreach materials (modified) The figure on the right is a manipulation of the figure on the left (using Photoshop) to make it match the fabricated data

Case Study #3 Subject: A graduate student. The allegation: He knowingly falsified data and experimental results in four manuscripts (three published) due to “a combination of lack of motivation, laziness, and a lack of interest in [his] work. ” ► Institutional investigation: In the course of the investigation, the university committee stated: ► ► ► “In our combined tenure, we have never before seen this level of intentional, knowing, long term fabrication of results. ” ► “The subject intentionally violated the honesty expected within the student-mentor relationship. ” Significance of the subject’s actions: The scientific record was affected and the papers eventually retracted. Prior to retraction, they had been cited 42 times. ► Actions by institution and NSF: ► ► His Masters degree was rescinded. ► The subject was debarred for a period of three years. ► For a period of three years after debarment, the university must certify that all proposals submitted by the subject do not contain plagiarized, fabricated, or falsified material. ► In all his future work, another researcher must vouch for the authenticity of the data. overhead from NSF-OIG outreach materials (modified)

Summary ► NSF-OIG is tasked with overseeing activities related to NSF operations by § Promoting economy, efficiency, and effectiveness… § Preventing and detecting fraud, waste, and abuse ► Its role is to ensure that the scientific record is protected and valid

References ► ► ► http: //oig. hhs. gov/fraud/complianceguidance. html http: //www. nacua. org/documents/Fed. Sentencing. Guidelines. pdf http: //www. ussc. gov/corp/Murphy 1. pdf http: //www. usdoj. gov/dag/cftf/corporate_guidelines. html Committee on Science, Engineering, and Public Policy. On Being a Scientist. Responsible Conduct in Research, 2 nd edition, p. 17. National Academy Press. Washington, D. C. 1995. Grant, G. ; Odell, G. , and Forrester, R. Creating Effective Research Compliance Programs in Academic Institutions; Academic Medicine, Vol 74, No. 9, September 1999, p. 951. Jordan, K. and Murphy, J. ; Compliance Programs: What the Government Really Wants. 1996, p. 121. Managing Externally funded Research Programs; A Guide to Effective Management Practices; Council on Government Relations, June 2005 Steneck, N. H. Introduction to the Responsible Conduct of Research. 2007, p 35. DHHS Draft OIG Compliance Program Guidance for Recipients of PHS Research Awards; Fed. Reg. Monday Nov 28, 2005, vol. 70 #227, p: 71312 Ingram, Ginna; Corporate Compliance Programs: More Than Window Dressing. Journal of Public Inquiry, Spring/Summer 2007, p. 5. http: //www. ignet. gov/randp/sp 07 jpi. pdf Office of Research Integrity, HHS. Sample Policy and Procedures For Responding to Allegations of Research Misconduct. http: //ori. dhhs. gov/policies/ori_policies. shtml