CCTS UIC Center For Clinical and Translational Science

CCTS UIC Center For Clinical and Translational Science SUCCESS CASE METHOD: APPLICATION IN A CTSA CONTEXT Nancy J Bates, Dr. PH, RD, CHES Timothy Johnson, Ph. D University of Illinois at Chicago

OVERVIEW SCM Steps Focusing and planning the SCM Creating an impact model Surveying for best and worst cases Interviewing & documenting success cases Communicating findings, conclusions and recommendations TL 1 Success Case Example Lessons Learned

The SCM Book

Background: SCM SIMILAR to other evaluation methods UNIQUE purpose is to help stakeholders understand • what worked, what did not, • what important results have been achieved • what can be done to improve program delivery and outcomes DOES NOT claim to be a comprehensive evaluation Focusing and planning the

Creating an impact model OUTCOMES ACTIVITIES Develop/ Implement formal program for TL 1 s and Predoctoral Support for Translational Research (TR) Program OUTPUTS SHORT-TERM # TL 1 s & TRs admitted/graduated TL 1 s & TRs /faculty feedback/evaluation # TL 1 s & TRs students supported by programs # programs developed LONGER-TERM Increased # of successful CTS researchers: Established TL 1 training program & TR program ↑ TL 1/TR placement in CTS track after grad ↑ CTS publications

SURVEY FOR BEST AND WORST CASES Brinkerhoff recommends creating surveys to find relevant cases Existing CCTS User data

Inclusion Criteria More/Less Successful Cases

Interviews: Brinkerhoff’s Main Questions What’s really happening? What results, if any, is the program helping to produce? What is the monetary value of the results? How can the initiative be improved?

TL 1 Selected Interview Questions 1. What are you career goals? How will the program help you get there?

TL 1 Selected Interview Questions 1. What are you career goals? How will the program help you get there? 2. In what ways is the program meeting (not meeting) your goals/needs?

TL 1 Selected Interview Questions 1. What are you career goals? How will the program help you get there? 2. In what ways is the program meeting (not meeting) your goals/needs? 3. How are (will) you use the training in your work?

TL 1 Selected Interview Questions 1. What are you career goals? How will the program help you get there? 2. In what ways is the program meeting (not meeting) your goals/needs? 3. How are (will) you use the training in your work? 4. What barriers do you experience?

TL 1 Selected Interview Questions 1. What are you career goals? How will the program help you get there? 2. In what ways is the program meeting (not meeting) your goals/needs? 3. How are (will) you use the training in your work? 4. What barriers do you experience? 5. What challenges in balancing your time with the program?

TL 1 Selected Interview Questions 1. What are you career goals? How will the program help you get there? 2. In what ways is the program meeting (not meeting) your goals/needs? 3. How are (will) you use the training in your work? 4. What barriers do you experience? 5. What challenges in balancing your time with the program? 6. What else would make the program more useful?

INTERVIEWING & DOCUMENTING SUCCESS CASES Semi-structured interviews Triangulation of data sources and methods Cross case analysis Atlas. ti software v. 6

INTERVIEWING & DOCUMENTING SUCCESS CASES Brinkerhoff’s Impact Profile Impact at a Glance Impact Story: Background and setting. Impact Story: Immediate outcomes. Impact Story: Organizational impact. Impact Story: What helped and what did not.

Documenting Case Example: Impact at a Glance: Outputs Are you reaching the audience you intended to reach? Yes # students admitted = 6 # students graduated = 2 Attrition: None Student feedback: Very grateful for the opportunity and pleased with the program.

Impact at a Glance: Long Term Outcomes ↑ TL 1 placement in CTS track after grad: Grad 1: Ph. D: Job in consulting and life science Grad 2: MD/Ph. D: matched a residency program that was her top choice “[I hope to be] at a research institute teaching, doing research and clinical work. So if I can couple all those, I mean, that's my absolute dream to do. ” ↑ CTS publications: MD/Ph. D grad: 2 published, writing others.

Impact Story: Background and Setting Program Description: This mentored TL 1 consists of. . . Program Director: Larry Tobacman, MD, . . . Target group: Basic biomedical Ph. D candidates and MD/Ph. D candidates who have a translational topic for their dissertation and are preparing for careers as translational researchers. Reach: Are you reaching the people you intended to reach? All meet the eligibility criteria. Attrition: None.

Impact Story: What Helped? Funding Allows them to focus on their research Educational fund gives them a sense of ownership Helps PIs tremendously since they don’t have to pay their whole stipend

Impact Story: What Did Not Help? Did not understand the need for a clinical mentor MD/Ph. D: clinical training, clinicians on dissertation committee BUT very excited to have a Ph. D advisor who did clinical work Ph. D: just getting to clinical research

Impact Story: What to Improve? Sense of TL 1 Community Didn’t meet the other UIC TL 1 s until St. Louis meeting Didn't know What the administrative roles of people were How the details of everything worked Defined the community as having a: Physical location for TL 1 s Go-to person Listserv Place to hang out together and have informal conversations about their work.

COMMUNICATING FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Brinkerhoff: Impact Profiles are often adequate in themselves Our experience: Generally true

LESSONS LEARNED �We found that incorporating the SCM with cross-case study analysis methods of Yin (2008) and Stake (1995) added to our overall understanding of program success.

REFERENCES Brinkerhoff, Robert O. The Success Case Method: Find Out Quickly What’s Working and What’s Not. Berrett-Koehler Publishers, 2003. Stake, Robert E. The Art of Case Study Research. Sage Publications, 1995. Yin, Robert K. Case Study Research: Design and Methods. 4 th ed. Sage Publications, 2008.

UIC Mascot Sparky D. Dragon

CCTS UIC Center For Clinical and Translational Science Nancy J Bates, Dr. PH, RD, CHES nbates@uic. edu CCTS Senior Evaluation Specialist Timothy Johnson, Ph. D, timj@uic. edu Director, CCTS Evaluation and Tracking Professor, Department of Public Administration Director, Survey Research Laboratory This project was supported by the University of Illinois at Chicago (UIC) Center for Clinical and Translational Science (CCTS), Award Number UL 1 TR 000050 from the National Center for Research Resources. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Research Resources or the National Institutes of Health.