The National Research Council (NRC) recently released a long-awaited report, Identifying and Supporting Productive STEM Programs in Out-of-School Settings. The National Science Foundation (NSF) sponsored this report as a follow-up companion study to the Successful K-12 STEM Education report that examined effective approaches to STEM education in schools.
As afterschool and other out-of-school-time STEM programs have grown in number over the past decade, the interest in measuring their effectiveness and impact has also grown. The recent America After 3PM study revealed that 10.2 million children participate in afterschool programs in the United States, up from 6.5 million a decade ago. Further, 69% of parents with children in afterschool programs say that some form of STEM activities are included in these programs.
NSF charged the Board on Science Education and the NRC Committee on Successful Out-of-School STEM Learning to conduct a landscape study and review and synthesize existing research in order to outline the criteria that policy makers, program developers and other stakeholders can use to identify effective out-of-school STEM settings and programs. More information is available on the project website.
The Committee found that the growth in programs has outpaced research on not only what types of programs have the best results for students but also what instruments should be used to measure just what those results are or should be. While the report acknowledges the challenge of evaluating diverse programs, it recognizes that learning occurs across space and time; experiences and knowledge accumulate as a result of this learning; and impact is not always limited to the student but is also reflected in program design and the community.
Consequently, evaluations of impact have to avoid depending entirely on short-term student outcomes and find ways to recognize more complex and longer-term outcomes. An extremely important recommendation the report also makes is that, while innovative evaluation approaches are needed to capture the outcomes, it is vital that such approaches do not end up formalizing the informal settings or disrupt the learning experience for young people.
The Committee discovered that the research and evaluation findings were not yet robust enough across the field to make generalized statements about which programs worked for which populations and under what circumstances. However, the findings are strong enough to identify three criteria that result in strong outcomes for students and the design needed to result in those outcomes:
ENGAGING - Engage Young People Intellectually, Academically, Socially and Emotionally
- Program provides firsthand experiences with phenomena and materials.
- Program engages young people in sustained STEM practices.
- Program establishes a supportive learning community.
RESPONSIVE - Respond to Young People’s Interests, Experiences and Cultural Practices
- Program positions STEM as socially meaningful and culturally relevant.
- Program encourages young people to collaborate and to take on leadership roles in STEM learning activities.
- Program positions staff as co-investigators and learners alongside young people.
MAKE CONNECTIONS - Connect STEM Learning in Out-of-School, School, Home and Other Settings
- Program connects learning experiences across settings.
- Program leverages community resources and partnerships.
- Program actively brokers additional STEM learning opportunities.
These criteria and the design characteristics should prove very useful for afterschool program developers and leaders as they offer very specific guidance on what programs can do. Many of these characteristics are extremely well-aligned with the philosophy of afterschool programs.
The report also discusses efforts made to create systems of support for afterschool and out-of-school STEM programs including the National Network of Statewide Afterschool Networks created by the Charles Stewart Mott Foundation and an initiative to increase collaboration between afterschool programs and local science centers undertaken by the Afterschool Alliance and the Association of Science-Technology Centers to increase access to more hands-on learning opportunities for students.
The report ends with recommendations for six actions that policymakers, program developers and stakeholders should consider for developing and maintaining productive programs:
- Understand the local conditions for community programs that support STEM learning.
- Design programs to achieve access, equity, continuity and coherence.
- Support the use of creative and responsive approaches to evaluate the success of programs at the individual, program and community levels.
- Increase the professionalization of out-of-school program leaders and staff.
- Strengthen the STEM learning infrastructure for the long-term.
- Invest in research to improve understanding of STEM learning in out-of-school programs.
While the findings in the report are not surprising, it is an extremely positive development for the afterschool field for an influential organization such as the NRC to highlight these issues in a document that will carry significant weight with policymakers and funders. The study makes a very strong case to think about learning in a more holistic way so that education reform is not limited to school improvement efforts but includes out-of-school programs. Recent policy developments to include afterschool as partners in STEM and the STEM ecosystems initiative launched by the STEM Funders Network all suggest that afterschool STEM is now on the map. Let’s take advantage of it!