Abt Associates Evaluation of Citizen Schools ELT Model
Dear Friends,
Since our earliest days at Citizen Schools, we have invested in rigorous external evaluation to understand the impact of our program on students, school communities, and Citizen Schools volunteers, and to identify the priority areas for improvement. Today marks an important milestone in this commitment. We are pleased to release the final report of a five-year external evaluation of the Citizen Schools national Expanded Learning Time (ELT) initiative, conducted by Abt Associates. I have below provided context on the study’s findings and shared how these results have influenced our work and strengthened our resolve to ensure all students thrive in the classroom and succeed as adults.
This evaluation includes 35 middle and K-8 schools across seven states that implemented Citizen Schools ELT for one or more years. The evaluation was designed to assess both fidelity of implementation and student impact relative to a set of carefully matched comparison schools in the same school districts that did not adopt the Citizen Schools program.
STUDENT IMPACT
The most pronounced student impacts are in the critical area of Social Emotional Learning (SEL). Based on student and teacher survey responses, Citizen Schools participants are significantly more likely to report that the program helps their self-esteem and pro-social behaviors. They are also more likely to participate in activities that help them learn about college and careers. These results are consistent with Citizen Schools’ longstanding goals of building students’ access to positive adult role models and career opportunities through project-based, real world learning opportunities. While the evaluation revealed consistently positive impacts on students’ academic achievement (as measured by state assessments in Math and English Language Arts) the overall results were not statistically significant. However, exploratory analyses do show statistically significant impact, equal to three additional months of learning gains in Math, after the first year of Citizen School implementation, as well as an additional three-and-a-half months of learning gains in Math for 7th grade participants specifically.
In the report’s conclusion, Abt Associates recommends that Citizen Schools build on these findings by investing in further measurement of the program’s impact on SEL, given their importance to students’ long-term educational and life success. We’ve partnered withTransformingEducation, a national leader in SEL research and practice, to do so, and will share more of that research later this year.
IMPLEMENTATION
In terms of implementation findings and lessons learned, the quality and depth of relationships between Citizen Schools staff and school staff is a critical lever in successful implementation.
Abt summarizes the other key implementation findings this way: “Overall, the ELT schools are clearly committed to implementing Citizen Schools’ ELT programming with fidelity, and at the same time, Citizen Schools has continued to recognize the need for flexibility with its campus partners. … Indeed, that variability seems to be essential for the model to be adaptable across such diverse contexts. The implementation findings also highlight some of the challenges associated with launching a multi-faceted model in dynamic settings, coupled with built-in staffing changes.” The other most frequent concern faced by schools and Citizen Schools is financial sustainability given the cost of the ELT model, particularly for schools and districts relying upon time-limited federal funding for turnaround or school improvement interventions like ELT.
Finally, we learned a great deal about the perceived value of the model’s component parts. School principals, who are critical to successful implementation and student success, value apprenticeships most.
ACKNOWLEDGEMENTS
I want to extend our appreciation to Abt Associates for their rigor and partnership, and to Citizen Schools’ staff and the leadership and faculty at our school partners for their participation in this evaluation. We also want to recognize the Edna McConnell Clark Foundation, the Wallace Foundation, and the ELT Impact Fund co-investors for their advice, financial support, and keen interest in this multi-year evaluation.
We encourage you to review the findings using the links below and to reach out with any comments or questions that you’d like to share with me directly.
EVALUATION FINDINGS INFORMING EMERGING STRATEGY
We are proud of what we have achieved in our ELT sites over the last five years and believe that, in select places, it has been a game-changing innovation for students and families. For this reason, we remain committed to the ELT model -- and its continuous improvement -- in those places. We have also learned that this model, with its cost and complexity, is not suited to every high need school and market. Given this, we are examining ways to isolate and scale the centerpiece of our model, apprenticeships. Our goal is to offer apprenticeship-style learning to markedly more students from low-income communities in order to impart the skills, access and beliefs required to thrive in schools and in the modern economy. We’re exploring a variety of cost-effective delivery mechanisms that include the expanded day and the traditional school-day classroom and will spend the next few months testing these ideas with our partners and developing a pilot prototype for fall 2017. We remain steadfast in our commitment to close the opportunity and achievement gaps for our students and to reimagine the way in which students orient to their world. Thank you very much for your support of us on this wonderful journey. I will be in touch with you to share more details on our emerging strategy and as always, welcome your ideas and feedback.
Best,
Emily McCann CEO