Cancer Disparities Research Partnership Program Process Outcome Evaluation

Cancer Disparities Research Partnership Program Process & Outcome Evaluation Amanda Greene, Ph. D, MPH, RN Paul Young, MBA, MPH Natalie Stultz, MS NOVA Research Company

Presentation Outline n Evaluation Purpose n Evaluation Timeline n n n Conceptual Framework n Evaluation Question Examples Data Collection and Analysis Evaluation Products 2

Purpose of Evaluation n Measure relevance, effectiveness, and impact of CDRP Program in a consistent fashion n Evaluation findings to identify Program attributes to be applied to other settings n Meaningful annual reports for stakeholders within and outside the NCI 3

Type of Program Evaluation n 3 -year, 9 -month evaluation n Process components • Is program being implemented as planned • Is targeted audience being reached n Outcome components • Were short-term and intermediate objectives achieved 4

Evaluation Timeline

CDRP Goal Build and stabilize research capacity in radiation oncology in community-based institutions that care for a disproportionate number of medically underserved, low-income, and racial and ethnic minority populations. 6

CDRP Program Conceptual Framework 7

Evaluation Design: Focus of Key Questions n Improvement and conduct of radiation oncology clinical research in community-based health care institutions serving large populations experiencing health disparities n Clinical trial participation by target n Influence of partnerships between awardee institutions and academic research centers n Influence of TELESYNERGY® n Influence of Patient Navigation populations 8

Clinical Trials Examples of evaluation questions • Did the CDRP Program result in increased clinical and translational research with populations experiencing cancer health disparities? • Have community-based clinical researchers improved their knowledge, attitudes, & practice (KAP) of radiation oncology research? Outcomes • Increased participation in clinical trials by target populations • Improved KAP by local researchers • Increased number clinical trials available at local site • Publications and presentations about findings by local researchers 9

Partnerships Examples of evaluation questions • What is the nature of collaboration between grantees & partners? • What is the influence of collaboration on clinical research and treatment outcomes at the grantee sites? Outcomes • Increased frequency and quality of communication between partners related to clinical research and patient consultation • Increased collaboration (clinical trials, research contacts, patient consultations) among academic and communitybased clinical researchers. 10

TELESYNERGY® Examples of evaluation questions • What is the influence of TELESYNERGY® on building partnerships, facilitating clinical research, and improving treatment outcomes? Outcomes • TELESYNERGY® facilitates patient access to care, clinical decision making and diagnosis. • TELESYNERGY® increases research collaboration between community sites and their academic partners report. 11

Patient Navigator Program Examples of evaluation questions • Has Patient Navigation facilitated participation of minorities in clinical trials, improved access to cancer care services, and impacted patient outcomes? Outcomes • Increased participation in clinical trials by target populations • Patients’ barriers to cancer care (e. g. , financial, cultural, transportation) reduced. • Patients adhere to treatment protocols. 12

Target Populations for Evaluation Data Collection n n Cancer patients from targeted populations experiencing cancer health disparities Principal Investigators, participating radiation oncologists, and awardee institution staff n Partners from academic research centers n Patient Navigators 13

Data Sources Archival Data n Program-related written documents • PI meeting minutes, organizational records and charts • Historical data on local cancer incidence, prevalence, mortality • Baseline data on clinical trials and research activities at participating sites New Data n n In-depth structured interviews with PIs and partners Patient focus groups Group interviews with Patient Navigators and Navigator support staff Surveys on issues related to recruitment to clinical trials 14

Comparison Group n n n 6 similar community-based radiation oncology facilities Comparison sites will be chosen based on same criteria as stated in RFA Conduct brief survey about independent and collaborative clinical research capabilities and current participation in radiation oncology clinical trials 15

Data Analysis n Each program component will be analyzed • Clinical trials • Partnerships between awardee and academic partner • TELESYNERGY® • Patient Navigation n Sources of information will be combined to analyze overall Program and linkages between/among components. 16

Data Analysis (continued) n Quantitative • Descriptive statistics • Complex analyses and causal modeling • Resource cost-allocation analysis n Qualitative—Thematic Analysis • • • Intended and unintended successes Failures Critical incidents Lessons learned Recommendations for future programs 17

Products of Evaluation: Reports n Annual process/progress reports • Program implementation, accomplishments, and progress to date • Needed midcourse changes to meet objectives. n Final outcome report • Evidence about Program’s attainment of goals and expected outcomes. • Summarize plausible mechanisms of change • Delineate temporal sequences between activities and effects 18