EVALUATION OF A TRANSDISCIPLINARY CANCER RESEARCH TRAINING PROGRAM

EVALUATION OF A TRANSDISCIPLINARY CANCER RESEARCH TRAINING PROGRAM FOR UNDERREPRESENTED MINORITY STUDENTS Michael T Halpern, SJ Dodd, Carolyn Y Fang, Yin Tan, Lin Zhu, Olorunseun Ogunwobi, Grace Xueqin Ma July 2, 2019 The views expressed in this paper are the authors’ and do not reflect the views of the U. S. National Institutes of Health or the National Cancer Institute In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem, Israel

Cancer Disparities in United States (U. S. ) • In the U. S. , racial/ethnic minority populations experience a disproportionate cancer burden. • • Increased incidence Increased diagnosis at advanced state Decreased access to timely and high-quality cancer screening and treatment Decreased survival • A well-trained, diverse scientific workforce is a key component in addressing health disparities; • However, investigators from diverse backgrounds are underrepresented in U. S. biomedical research.

Addressing lack of cancer researchers from under-represented populations: “The Partnership” • Temple University-Fox Chase Cancer Center/Hunter College (TUFCCC/HC) Regional Comprehensive Cancer Health Disparity Partnership (“The Partnership”): recently-initiated multi-institutions collaboration funded by the U. S. National Cancer Institute (NCI) • The Partnership is focused on training individuals from under-represented populations (including racial/ethnic minorities and first generation college students) in cancer research. • Involves a stewardship model that broadens scope of cancer research training to create new career pathways; prepares students and junior faculty to participate in multidisciplinary endeavors; and engages scientists more fully in society. • Also provides support for research activities focused on addressing cancer disparities and research funding for graduate students, post-doctoral fellows, and early stage investigators (junior faculty) through an internal competitive grant proposal process.

The Partners • Temple University (TU): Comprehensive research university founded in 1884 in Philadelphia including more than 40, 000 students (approx. 46% racial/ethnic minority); more than 2200 full time faculty; and a 583 -bed teaching hospital; • Fox Chase Cancer Center (FCCC): NCI-designated Comprehensive Cancer Center in Philadelphia, providing care to more than 69, 000 patients per year and including more than 325 doctoral-level researchers. Part of the Temple University Health System; and • Hunter College (HC): Comprehensive teaching and research institution in New York City serving approximately 23, 000 students (70% are ethnic/racial minorities, 28% first-generation college students) and more than 700 full time faculty.

TUFCCC/HC Partnership Structure – 5 cores, each with co-leaders from TUFCCC and HC: • Administrative Core: administrative, scientific and programmatic oversight and monitoring; • Biostatistics and Bioinformatics Shared Resource Core: enhances Partnership research by providing expertise in statistical analysis, bioinformatics, genomics, data management and statistical computing; • Community Outreach Core: develop and implement effective community outreach activities and interventions to reduce cancer disparities among underserved populations; • Research Education Core: support training of a diverse workforce to meet biomedical, behavioral and clinical research needs to address cancer health disparities; • Planning and Evaluation Core: evaluate short-term and long-term process and outcomes for the overall Partnership and each Core and provide recommendations to leadership to accomplish Partnership goals.

Partnership Training Activities • Includes training opportunities for undergraduate and graduate students, post-doctoral fellows, and early stage investigators (e. g. , junior faculty). • Summer Cancer Research Institute: 8 -week research intensive program; students to participate in a cancer curriculum and gain hands-on training in cancer research; • Mentored Research Projects: trainees match with mentors to receive experience in developing projects, conducting research, analyzing data, and reporting outcomes; • Includes postdoctoral fellows and research associates as “peer mentors”; • Enriched Training Activities: cancer research seminars, journal clubs, professional/skills development workshops, assistance with conference presentations.

Evaluation of the Partnership • A requirement for the NCI funding is to perform a comprehensive evaluation of the Partnership. • How to structure an evaluation of a multi-institution, multi-core, partnership that includes a broad range of training activities, research support, and research funding for individuals ranging from undergraduate students through senior faculty? • In addition, need to be mindful of time required to submit evaluation measures. • Removing barriers for Partnership member to submit evaluation measures will increases quality and completeness of collected information as well as reduce participant burden and increase satisfaction with the evaluation process

Evaluation Logic Model for The Partnership Goal Administrative Core Provide effective leadership, integration, and organizational infrastructure Inputs/Resources Partnership Expertise • Institution Leadership Administrative Core • Provide support to U 54 Program • • Community Outreach C. Strengthen CBPR infrastructure for TUFCCC/HC Partnership & foster community engagement in cancer research Research Edu. Core Develop and support programs to increase diversity of cancer health disparities research trainees. Research Projects Full Study 1: Sustain long -term adherence of HBV care guidelines among underserved Asian Americans with CHB Full Study 2: Establish the relationship of informative genetic markers among African Americans for lung cancer Pilot: Establish the relationship Epigenetic Factors and the Microbiome in Colon Cancer Disparities among African Americans Biostat & Bioinfo Core Provide support in research design, data management and statistical computing, bioinformatics, epidemiology, genomics. Activities • • Engagement and Support Advisory Groups -EC -SILC -IAC -PSC -Comm Advisory Board Community involvement Coordination & integration among Cores/Projects and two institutions Integrated learning environment Researchers, URM trainees and support staff Partnership Eva. Data • Annual reports • Interviews with • • • Funding & Supplements) Non-CRCHD/NCI Grant Funding TUFCCC &HC Institutional Committed funds /resources Existing infrastructure & long-term partnership Existing data and research environment NCI/CRCHD Program Leadership Oversight & Guidance Administrative Core Short-Term • • Effective communication and management procedures/system Compliance between milestones and timelines Community Outreach Core • Improved knowledge and access on cancer screening • Strengthened community partners’ skills in cancer research • Successful participant recruitment and retention in research projects Research Education Core • Recruit and enroll diverse trainees in cancer education programs • Mentor and support enrolled trainees to achieve successful outcomes Research Projects Community Outreach Core • • • - TUFCCC/HC leadership - PSC members - TUFCCC/HC mentors & mentees • Meeting minutes • Progress tracking & monitoring data Resources • CRCHD/NCI Funding (U 54 cores/projects Monitor core/project progress & fiscal disbursement Facilitate and coordinate integration between TUFCCC and HC, as well as among cores/projects Outcomes----Impacts Engage community in cancer education initiatives Implement community wide cancer outreach activities & disseminate NCI cancer information Support Res. Projects in participant recruitment & engage REC mentees in CBPR community cancer research Research Education Core • Recruit URM ESIs and students • Provide mentorship and resource support to diverse ESIs and students Research Projects • Conduct funded research projects • Disseminate/scale up new projects Biostatistics & Bioinfomatics Core • • Provide study consultation, research support, and informatics tools Coordinate and arrange genomic and epigenetic services Full Study 1: improved long-term (24 -m) HBV care behavior improved motivation and self-efficacy for HBV care Full Study 2: Identified genetic markers for tobacco susceptibility and addiction in African American/Blacks Pilot Study: Identify cell lines for in vitro Ch. IP experiments Validation of Ch. IP antibody specificity Biostatistics & Bioinfomatics Core • Created Web portal interface for facilitating the process to solicit, manage, review and support new research projects • Constructed databases for collecting data generated from each Core/project Long-Term Administrative Core • Strengthened U 54 Partnership in HC cancer research capacity • Strengthened U 54 Partnership in TUFCCC cancer disparities research Community Outreach Core • Established solid cancer outreach infrastructure at HC • Increased cancer screening and cancer intervention participation in target communities Research Education Core • Demonstrated advancement of trainees at various cancer-related career and academic levels • Increased number of grant applications or successful submission and grant awards received by ESIs Research Projects • Inform intervention dissemination strategies and scaling up for larger trial to reduce cancer disparities; increased cancer disparity research projects • Increased grant awards on new full/pilot projects on cancer disparities Biostatistics & Bioinfomatics Core • • Established Biostatistics and Bioinformatics Resources Strengthened research capacity in Biostatistics and Bioinformatics • Sustained TUFCCC/HC Partnership to support cancer disparity research and improve diversity of cancer research workforce Enhanced cancer research capacity of community partners Disseminated evidenced intervention and new discoveries from cancer disparity research • • Impact

Evaluation of the Partnership (con’t) • Evaluation Framework: • Separate evaluation for each Core and for the overall Partnership to assess integration and synergy • For each component evaluation, assess: 1. Goals for the main activities; 2. Specific Objectives to determine whether each Goal is successfully accomplished; 3. Measures and Indicators to assess progress towards completion of each Objective; 4. Data sources for each measure/indicator

Research Education Core Evaluation Matrix

Overall Partnership Evaluation Matrix

Methods for Evaluation of the Partnership • To minimize evaluation data collection burden, use a real-time framework. • Gather evaluation data as activities being evaluated occur • Permits collection of evaluation information with enhanced efficiency and quality • Use of a web-based central evaluation platform facilitates data collection with minimal burden. • Allows decentralized reporting of measures/indicators. • Permits rapid generation of real-time reports and dashboards to assess Partnership progress and identify potential barriers to accomplishing Partnership goals • Facilitates feedback on Partnership services, workshops, seminars, and other supports and events via web-based anonymous satisfaction surveys

Web-Based Central Evaluation Platform https: //u 54 speech. org/ Home Page

Web-Based Central Evaluation Platform Cores Projects

Common Reporting for All Cores/Projects Report

Additional Reporting for Cores/Projects Report

Planning and Evaluation Core Accomplishments • Consulted with each of the 4 Cores to finalize their Core specific aims and objectives • Consulted with each of the 4 Cores to develop Core specific evaluation plans and metrics • Collaborated with the Administration Core to develop the Request for Applications for pre-pilot and pilot research proposals • Developed and submitted evaluation abstracts to 2 conferences • Participated in National U 54 PACHE conference calls to develop evaluation metrics across all U 54 Partnerships

Overall Accomplishments of Partnership 11 Publications; 32 Presentations (at professional conferences) 10 Grant applications submitted 76 Trainees/mentees engaged in cancer research 47 Mentors committed

Funded Pre-Pilot and Pilot Projects • • Distributed Two RFAs: Pre-Pilot (Graduate Trainees and ESIs); Pilot (ESIs) Received: 24 Letters of Intent 19 Applications Received: 10 pre-pilots, 9 pilots 9 from HC; 10 from TUFCCC 12 Biomedical Cancer Disparity Research 7 Behavioral Cancer Disparity Research Used NIH rigorous review process and criteria 2 External Scientific Reviewers for each application & provided critiques Selection committee met and consulted with IAC Selected 15 awards for funding based on scientific merit (7 pre pilots and 8 pilots)

PEC Challenges and Proposed Solutions • Ensuring that appropriate evaluation objectives, measures and data sources were identified for each core. • Addressed by consistently meeting with each set of Core leaders until measures and data sources reflecting their aims were clearly identified. • Going forward, will continue to monitor the data sources to ensure they do capture the aims as expected. • Developing comprehensive database to centralize/standardize data collection and analysis. • Web-based portal and centralized database now available.

PEC Goals/Milestones for Next Year • Test the centralized database for comprehensive data capture and reporting. • Begin analysis of evaluation data. • Continue to provide feedback to the Cores and leadership regarding progress towards Partnership goals and objectives. • Review and revise each core’s and overall partnership’s evaluation measures and data sources.

Acknowledgement • The project described was supported by TUFCCC/HC Regional Comprehensive Cancer Health Disparity Partnership, Award Number U 54 CA 221704(5) from the National Cancer Institute. • The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health.

QUESTIONS? In. SITE 2019 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 30 - July 5, 2019 Jerusalem, Israel