Airborne Astronomy Ambassadors program evaluation Advance National Educational

Airborne Astronomy Ambassadors program evaluation Advance National Educational Goals / Support Co-STEM goal achievement by 2020: • Project Outcome: AAA is aimed to measurably enhance student STEM achievement & engagement via teacher professional development in selected school districts. • Outcome Metrics: Thirty-eight teachers have been randomly assigned by West. Ed to either “treatment” or “control” groups; their students will be tested on STEM knowledge at the beginning and end of each teacher’s course (pre- & postassessments). Results will be analyzed using Hierarchical Linear Modeling to assess the impact of teacher AAA participation and the special AAA IR curriculum on student performance, content knowledge, and STEM engagement.

AAA Evaluation, cont’: Sharing the Science - the “Big Idea” about this study • Important reminder: AAA is the only Sci. Act Cooperative Agreement that is about formal teacher professional development resulting in changes to student instruction. • West. Ed will evaluate the impact of AAA by conducting a Randomized Controlled Trial (RCT) of the program, in which West. Ed has assigned 23 teachers (~ 2000 students) to ‘treatment’ and 15 teachers (~1500 students) to ‘control’ (delayed treatment), measuring their students. • Important lessons learned from the study so far: 1. Offering a delayed treatment (letting the control group participate in the program in 2018– 2019) is vital for gaining the buy-in of a control group for an educational RCT. 2. It’s important to avoid a “black box” evaluation when doing an RCT in evaluation; it’s critical to collect information on both treatment and control conditions to interpret results.