Using Logic Models to Enhance Collaboration Between Internal

Using Logic Models to Enhance Collaboration Between Internal Evaluators and Program Staff: Reflections from the Evaluation of A STEM-Based Youth Program Cassandra L. Tavaras AEA Conference November 2017

MGH Youth Scholars Program Began with HS Class of 2012 120 high school youth in grades 9 -12 from Boston area participate each year in weekly program STEM focused activities/workshops and career exposure

Determining Need for Revised Logic Models Shifts in program focus and growth Strengthen commitment to datadriven recommendations

Program Staff-Evaluator Relationship Prior to Revision Process Ideal first project Previous evaluator for MGH Youth Programs was not as active with staff Client-evaluator relationship

Revision Process Determination of topic areas: -College Knowledge -STEM Education/Exploration -21 st Century Skills Program Managers reviewed and compiled research on focus areas and presented to rest of team/evaluator Team reacted to research and discussed attainable program outcomes for each area Team drafted language for indicators/outcomes and worked side by side with evaluator to revise language and think through possible data sources/tools together

Program Staff-Evaluator Relationship During Process Opportunity to build on staff initiative for process and drive for creating new logic models to establish partnership Collaborative nature-all perspectives were considered and reflected upon Entire process lasted about 8 months

High-Level Program Logic Model 9 th & 10 th grade 11 th & 12 th grade College & Beyond

Results of Revision Process Renewed sense of purpose and re-energized staff in thinking about the role of evaluation Tailored evaluation activities to reflect process and new outcomes Engaging in this process enhanced the relationship with evaluator-equal, integral thought partner Evaluator brought in more often which promotes continuous understanding/knowledge of crucial program pieces to facilitate ongoing evaluation and improvement

Maintaining Staff Buy-In

Lessons Learned/Key Takeaways Patience is key Approach to relationship is important to establish rapport and trust Importance of demonstrating the data/evaluation in action

Thank You! Cassandra Tavaras Project Manager, Evaluation and Strategic Support MGH Center for Community Health Improvement Email: ctavaras@partners. org