CITI RCR Developer Group Seattle WA October 24

CITI RCR Developer Group Seattle WA October 24, 2009

Survey conducted 11/2004. Presented 3 -11 -05. CITI Developer Meeting, Key West, FL

CITI RCR Program • Initiated in 2005 • $50, 000 from ORI 9 -1 -05 to 6 -01 -06. – Public access RCR program. – Continue for as long as ORI funds the RCR help desk. • $50, 000 12 -08.

CITI RCR Course Content • The RCR in the: – – Biomedical sciences. Social and behavioral sciences. Physical sciences. Humanities. • Textual material similar from course to course. • Embedded Case studies are discipline specific.

CITI RCR Course Content Discipline specific courses Textual materials and case studies are discipline specific. • RCR for science administrators – Steve Erickson Ph. D. and Karen Muskavich Ph. D. Boston College • RCR for engineers and engineering students. – Jason Borenstein Ph. D. - G. I. T. • The RCR in the humanities. – Sharon Shriver Ph. D. – Penn State University – 1 -1 -10 • The RCR in the social and behavioral sciences – Dan Vasgird Ph. D. Spring 2010

New additions to the RCR Program

Lab animal welfare

Human subjects protections

RC Scholarly Activities in the Humanities

Modifications to the SBR RCR modules

New additions to the Program Introductory video cases • Introductory video cases - 7 – 3 new “introductory” video case studies for SBR RCR course. • • • Misconduct Data Acquisition Conflicts of Interest

Content Editing • Content edits from Key west meeting have been implemented. • Hired a science and technology editor to resolve outstanding issues in the CITI Biomedical RCR modules. – Typos, misspellings, poor usage in the modules. – Grammatical errors. – Poor quiz question. • • • Deliver by November 14. Reviewed and reconciled by December 1, 2009. Implemented by 1 -1 -10.

RCR Course for Scholars in the Humanities Sharon Shriver, Ph. D. Pennsylvania State University

Humanities RCR course revision • Goals: • Retain the overall structure and some of the content of the existing CITI humanities course. • Omit science-specific content and add new content that is relevant for scholars in the humanities. • Use examples and case studies that illustrate the application of RCR principles in humanities settings • Include recent events (e. g. prominent cases of plagiarism in history, fraud in art) as examples of scholarly misconduct.

Humanities revision– timeline • November 15, 2009 ▫ All Modules complete ▫ All case studies and quiz questions revised ▫ Peer Review • December 15, 2009 ▫ Modifications implemented ▫ Final proofreading completed, post for review • January 1, 2010 ▫ Revised Humanities course available to users

Other new projects • Bio-safety and Bio-security – 15 modules • Plagiarism Module – Gary Comstock Ph. D. and PB • RCR in the physical sciences and mathematics.

Analysis of Web-based learning Released May, 2009 Meeting CD Informational Items and Resources

D. of Ed. Report • Analysis of 99 studies between 1996 and 2008 comparing online vs. classroom learning. • Some grade or high school setting, but, most done in colleges, universities, med schools and the military. • Students doing some or all on line ranked in 59 th performance %tile. • Students doing just classroom ranked in 50 th performance %tile. – “Most of the variations in the way online learning was used did not significantly affect student learning outcomes. ” – “The effectiveness of online learning approaches appears quite broad across different content and learner types. ”

Course site usage. • Program Growth – 119 - New institutions since 1 -1 -09; total of 1324. – 70 New institutions since, June 1, 2009 – Estimated FY 2010 ~ 130 – 86 New RCR set-ups since 1/2009

Course site usage Number of NEW Learners per month completing at least 1 CITI course 60 000 9 -2009* 50 000 40 000 30 000 20 000 10 000 0 May 2004 Nov 2005 May Nov 2006 May Nov 2007 May Nov 2008 May Nov 2009 May 2010

RCR Course Completion Reports (C. R. ) awarded Year 2009* 2008 Sites 222 (45%) 136 23, 361 18, 849 C. R. /month 2596 (65%) 1571 C. R. /day 87 (~1200) 52 Total C. R. * 1 -1 -09 to 10 -1 -09, public access and institutional RCR set-ups * 86 new institutional RCR set-ups, 2009 CRs ALL DISCIPLINES RCR CRs / ALL CRs 328, 999 330, 227 6. 8% 5. 6% 10 -1 -09

RCR Course Completion Reports (C. R. ) Awarded via The Public Access Portal. Public Access (PA) only 2009* P. A. Registrations 7152 (795/m) 64% P. A. SBR 1417 P. A. biomed 960 PA - Humanities 203 PA - PSE 157 Total PA C. R. 2737 (39%) PA_C. R. /month 304 (12%) * 1 -1 -09 to 10 -1 -09 2008 5835 (486/m) 1273 935 324 94 2626 219 (14%)

Top 10 institutional RCR users 2008 – 136 ; 2009, 222 sites; = 45% increase 1 2 3 4 5 6 7 8 9 10 Institution C. Rs Public Access Wayne State University-Detroit MI Purdue University of California Los Angeles Oklahoma State University Vanderbilt University of Miami/Jackson Health System Clemson University of Wisconsin - Madison Cleveland Clinic Foundation Pennsylvania State University 2850 2670 1687 1167 1053 601 593 497 456 444 370 10 -14 -2009 all CRs 24, 470

Research on the RCR Mentoring Participation in HSR

Experiences of Mentors and Mentees Survey Paul Braunschweiger, Ph. D. Ken Goodman, Ph. D. Sergio Litewka, MD CITI Program University of Miami Ethics Program

Working Hypotheses • Although mentoring is crucial to the social and professional development of students and young investigators, mentoring programs are not wide spread in academia. • The role of Mentor and Research advisor is often confused. • Many people who mentors may be ill prepared for the job.

Conclusions • Data management is the most common research ethics topic discussed by mentors and mentees – >25% of Students say that discussing research ethics with mentor is a rare event. – Only 5% of mentors say that discussing research ethics with student is a rare event. • Rather low % of people mentoring students (~30%) say they have specific training for mentoring.

Conclusions • For ~60% of students, the mentor and res. advisor are the same person. • Only about 20% of responders indicated that their institution has a mentoring program. • Majority of survey subjects didn't know if their institutions had a mentoring program. – most pronounced in student groups

CITI RCR Satisfaction Surveys RCR Course Satisfaction Survey 9 -1 -08 to 10 -1 -09

Which RCR course did you complete? 50% 45% % Responders; n=8326 40% 35% 30% 25% 20% 15% 10% 5% 0% The Social and Behavioral Sciences RCR Course The Biomedical Sciences RCR Course The Physical Sciences RCR Course The Humanities RCR Course. The RCR Course for Research Administrators n=10784

I completed the CITI RCR Course because: % Responders; n=7179 40% 38% 35% 30% 28% 25% 20% 15% 11% 10% 5% 4% 2% 2% 0% Reviewer Recommended by mentor or colleague. Undergraduate requirement Required by employer.

70% Have you completed RCR instruction previously 68% % Responders; n=9367 60% 50% 40% 30% 25% 20% 10% 3% 4% 0% No, my Mentor has provided guidence Yes, in a classroom Yes, another on line course No, this is new to me.

The RCR modules present a short video case study as an introduction to the module. Please provide us with some feedback regarding these "Introduction" videos. Choose as many as is appropriate to convey your opinions. The 3 minute video case studies were not long enough to introduce the topic thoroughly. The Introductory video case studies were not well done. 1% The Introductory video case studies were not very helpful. 3% The 3 minute video case studies were too long. 5% CITI should make introductory videos for the other RCR topics. It's all about economy of time. I prefer text, because I can read faster than I can listen. The Introductory video case studies were well done. The length of the 3 minute video case studies was about right. The Introductory video case studies were helpful to frame the content before I started reading. 2% 8% 9% 23% 24% 25% 85% OF RESPONDERS, N=12743, VIEWED VIDEOS

Which of the RCR modules was most informative or most helpful to you? Collaborative Research Responsible Peer Review Data Acquisition and Management Responsible Authorship Mentoring Conflict of Interest Research Misconduct 7% 7% 12% 13% 15% 17% 29%

Case studies are thought to be an effective approach for adult learners. Most of the modules contained short case studies imbedded in the text (folder icons). Did you review any of these case studies. 50% 45% 40% 35% 30% rs; n=8982 25% 20% 15% 10% 5% 0% Reviewed at least one per module Didn't review any Reviewed all Reviewed a few

The case studies were very useful to put the ethical concepts described in the foundation text into a practical "real life" context. 25% 21% 20% 17% 16% 15% =6655 11% 10% 5% 5% 4% 4% 3 4 2% 0% 1 2 Strongly disagree 5 6 7 8 strongly agree 9

After completing the CITI RCR Course, I have a better understanding of the ethical issues central to the research enterprise so that I can conduct my work with the highest level of integrity. 35% 32% 30% % Responders; n=8771 25% 22% 18% 20% 15% 10% 5% 4% 2% 2 3 4 9% 8% 5 6 0% 1 Strongly disagree 7 8 9 strongly agree

Now that I have completed the CITI RCR Course, I am more confident in my ability to advise a student or a colleague on an issue involving the Responsible Conduct of Research. 25% 22% 20% % Responders; n=6821 15% 11% 5 6 10% 5% 3% 2% 3% 3% 2 3 4 0% 1 Strongly disagree 7 8 9 strongly agree

Having completed the CITI Course in the Responsible Conduct of Research and gained a better understanding of how damaging research misconduct can be to students, to the institutions and to other investigators, I am now more likely to report a clear act of 30%research misconduct to an appropriate person at my organization. 29% 25% % Responders; n=8267 20% 15% 13% 9% 10% 5% 19% 3% 2% 2% 3% 3 4 0% 1 2 Strongly disagree 5 6 7 8 9 strongly agree

Rate the course. RCR 27% 30% L. A. W. 30% 28% 25% % Responders; n=8390 21% 20% 17% 20% 15% 12% 13% 15% 21% 12% 15% 10% 5% 2% 2% 3% 3% 5% 1% 1% 2% 2 3 4% 0% 0% 1 2 3 4 5 6 7 Poor 8 9 Outstanding 28% 30% 1 Poor 22% 25% 20% 17% 12% 13% 15% 10% 5% 1% 1% 2% 3% 3 4 0% 1 Poor 2 5 6 7 8 9 Outstanding H. S. P 4 5 6 7 8 9 Outstanding

The Future of Responsible Conduct of Research Education John Galland Ph. D. Director, Division of Education & Integrity Office of Research Integrity / DHHS

15 minute Break resume at 10: 50

Mentoring in Engineering & Non-Medical Sciences Suzanne Brainard, Ph. D. Professor, Human-Centered Design & Engineering Executive Director of the Center for Workforce Development University of Washington

The Responsible Conduct of Social and Behavioral Research Dan Vasgird Ph. D.

CITI International RCR Initiatives Sergio Litewka MD

The Engineering and Science Issues Test (ESIT): A Discipline –Specific Approach to Assessing Moral Judgment Jason Borenstein Ph. D.

Lunch Resume at 1: 30 pm

Rate the course. RCR 27% 30% L. A. W. 30% 28% 25% % Responders; n=8390 21% 20% 17% 20% 15% 12% 13% 15% 21% 12% 15% 10% 5% 2% 2% 3% 3% 5% 1% 1% 2% 2 3 4% 0% 0% 1 2 3 4 5 6 7 8 9 1 28% 30% 22% 25% 20% 17% 12% 13% 15% 10% 5% 1% 1% 2% 3% 3 4 0% 1 2 5 6 7 8 9 H. S. P 4 5 6 7 8 9

Reviewing and editing of RCR course materials The case studies

Meeting CD

Group Leaders • Marianne Elliot MS – Research Misconduct • Paul Braunschweiger Ph. D. – Collaborative Research • Susan Miller – Peer review • Jason D Borenstein Ph. D. – Data Acquisition and Management • Ruth Fischbach Ph. D. – Conflict of interest • Iekuni Ichikawa M. D. – Authorship • Dan Vasgird Ph. D. – Mentoring

Group Leader Reports • Marianne Elliot MS – Research Misconduct • Paul Braunschweiger Ph. D. – Authorship • Peer review – Peer review • Jason D Borenstein Ph. D. – Data Acquisition and Management • Ruth Fischbach Ph. D. – Conflict of interest • Iekuni Ichikawa M. D. – Collaborative Research • Dan Vasgird Ph. D. – Mentoring • Mike Mann Ph. D. – Lab Animal Welfare

Thank You Please send in revisions, edits and modifications by Nov. 1, 2009.