Phillips Brooks House Association PBHAs Summer Urban Program

Phillips Brooks House Association (PBHA)’s Summer Urban Program Overview: PBHA is a student-run non-profit with a dual mission of student development and community impact. PBHA youth programs build positive relationships with youth, families, and community partners to ensure that Boston and Cambridge youth ages 5 -19 can build the social, emotional, and academic skills needed to define, access, and achieve their own success in and beyond our programs. Site info: 12 public school buildings in Boston and Cambridge Program Type: Mix of academic and enrichment Students Served: 920 youth, 5 -18 year olds Staff: all sites combined have 130 FT seasonal college students, 10 FT year-round professionals, 80 FT seasonal high school students

Why Evaluation • 423 Staff Applications • 910 Youth • 129, 647 hours of programming …. But wait, there’s more! 5 Long-term Outcomes Academic, Support System, Future, Community Engagement, Self-Worth Staff Management Training, Coaching, Data Collection Processes

Objectives and Agenda • Building an Inquiry Approach • Evidence-Based Improvement Process – 1. SAYO – 2. Writing Rubric – 3. APT Observation Data • Individual Program Data • Debrief and next steps

Ex 1

Ex 2

1 = Strongly Disagree; 3 = Neutral; 5 = Strongly Agree Ex 3 (n=6)

1 = Strongly Disagree; 3 = Neutral; 5 = Strongly Agree Ex 4 (n=6) (n=40)

Validity • Reliability: measurement tool design that is consistent each time it’s used • Sampling: the questions relate well to the construct and representative group answered the questions • Scaling: scores using scaled answers lead to more or less significant differences between two or more groups of results

Plan highest leverage strategy Identify a problem we control Focus on Evidence and inquiry into data What more data do you need? What do you wonder? (focus questions) What do you notice? (evidence) Program matic decisions and approach based on data

Case Study 2014 Benchmark average for each index = 3. 0 (Mostly True)

2014 APT Averages by Domain (out of 4) n=42 (no Middle Up)

Data Transparency/Sharing • Google Presentations • Infoactive: – https: //infoactive. co/plays/32998 • Tableau Public