BPCA Program Evaluation Plan Leslie Goodyear EDC Gary

BPC-A Program Evaluation Plan Leslie Goodyear, EDC Gary Silverstein, Westat January 14, 2013

The Evaluation Team EDC Team • Leslie Goodyear • Carrie Parker • Katrina Bledsoe • Alyssa Na’im • Tracy Mc. Mahon • Craig Hoyle • Lois Joy Westat Team • Gary Silverstein • Joy Frechtling • Liam Ristow Kansas State • Linda Thurston

Purpose of the Evaluation • The purpose of this program evaluation is twofold: 1. To maximize the past, present and future evaluations of the BPC Alliances 2. To investigate the aggregate impact and effectiveness of the BPC Alliances as a national resource for achieving the transformation of computing education for diverse student groups

Evaluation Questions Four broad Evaluation Questions • What is the impact of the BPC-A program on participants? • What is the impact of the BPC-A program for creating sustainable changes in culture and practices at the institutional, departmental, and organizational levels? • What is the impact of the BPC-A program on national trends related to computing? • What is the impact of the BPC Alliances, in aggregate, for providing a national infrastructure for achieving the transformation of computing education for diverse student populations?

Overview of Approach • Multiple and mixed methods to address each of the Evaluation Questions. • Two main parts: ▫ A retrospective look at the activities and outcomes of the Alliances for the first five years of the program, from 2006 – 2011 ▫ Evaluation activities that will document and evaluate the program for its second five years, from 2011 – 2015.

Data Collection Activities • Document Review • Monitoring Tools • Annual surveys of student and faculty participants • Cohort survey to follow students longitudinally • Case studies that will also include interviews and site visits • Cross-case analysis

Logic Model of the BPC-A Program

Timeline

Advisory Group

Alliance Involvement • • Alliance Evaluators Group; multiple meetings Consultation on timing of data collection Customization of data entry tools Technical assistance for monitoring and data collection • Distribute final evaluation plan

Questions? Thank You!! Leslie Goodyear 617 -618 -2354 lgoodyear@edc. org Gary Silverstein 301 -251 -2244 SILVERG 1@WESTAT. com