NASA's Airborne Astronomy Ambassadors Program: Evaluation of Student Outcomes

The Airborne Astronomy Ambassadors program aims to measurably enhance student STEM learning & engagement in selected school districts via professional development (PD) for high school science teachers. AAA teacher PD culminates with a week-long STEM immersion experience at NASA's science research aircraft facility in Palmdale, California & participation in research flights on NASA's Stratospheric Observatory for Infrared Astronomy (SOFIA).

AAA program staff produced a 10-day electromagnetic spectrum & multi-wavelength astronomy (EMS/MWA) high school curriculum module for AAA teachers to incorporate into their science courses. This module is aligned with Next Generation Science Standards (NGSS). The SETI Institute contracted with WestEd evaluators to perform a randomized controlled trial (RCT) during the 2017-18 school year to assess the impact on student outcomes of AAA teacher PD & subsequent delivery of the curriculum module. The RCT compared student content learning and STEM engagement for a treatment group of 1473 students receiving the curriculum module from teachers who had already flown on SOFIA with a control group of 870 students receiving "business as usual" electromagnetic spectrum instruction from teachers who had not yet flown on SOFIA. The parent population of 38 teachers were randomly assigned to treatment versus control groups.

The treatment group's mean post-curriculum content test score was a factor 1.18 higher than that of the control group, a difference of more than 9 standard errors. This result is remarkable because: (1) the treatment group's teachers were delivering the AAA EMS/MWA curriculum module for the first time, without a practice run; (2) the control group's teachers were expert instructors teaching well-established curricula; (3) student post-assessments were administered at the end of the respective academic terms, not immediately after the curriculum module was taught, thereby measuring medium-term content retention.

Funded by NASA Cooperative Agreement NNX16AC51