Chapin Hall

Data Use in Afterschool Systems Requires Investments in People and Process

Afterschool systems and their funders are looking to data to answer questions about program quality, youth participation, and program effectiveness. However, in this field, meaningful and actionable data are limited. Chapin Hall researchers leveraged the experiences of afterschool system staff, administrators, and other stakeholders in nine cities to identify strategies to build their capacity to use data to improve their systems.

What We Did

Chapin Hall conducted a four-year qualitative study of the data use in afterschool systems in nine cities. Each partner site was also participating in the Wallace Foundation’s Next Generation Afterschool System-Building Initiative. The nine cities were Baltimore, MD, Denver, CO, Fort Worth, TX, Grand Rapids, MI, Jacksonville, FL, Louisville, KY, Nashville, TN, Philadelphia, PA, and Saint Paul, MN.

Chapin Hall conducted site visits and interviews with key stakeholders, observed selected training and meetings related to data use, analyzed city documents, and attended cross-city grantee meetings.

What We Found

Context matters. Afterschool systems varied in what data they collected because of differences in city priorities, funding, and afterschool system goals. However, in general, afterschool systems invested in data systems to:

• Understand program supply and demand, quality, and effects on youth
• Manage programs and make decisions about program quality
• Improve how resources and services are allocated to address community contexts

A number of factors affected data use in afterschool systems:

• Norms and routines, partner relationships, leadership and coordination, and technical knowledge directly influenced staff’s data use in city afterschool systems.
• Building capacity and engaging stakeholders in data use required intentional strategies, time, and flexibility.
• Implementing program quality improvement systems delivered useful data on program strengths and areas for improvement. It also helped build capacity for and commitment to data use among frontline providers.

Afterschool systems’ ability to use data evolved over time. As their data systems became more established, they moved from mainly training staff on how to enter data to developing their skills in interpreting and applying data to make decisions and guide system growth.

.

What It Means

Developing the capacity to use data depends as much on people and processes as technology. Building the capacity to use data is not easy or straightforward. It requires time, patience, persistence, and flexibility. For our partner sites, starting small with a limited set of goals or a pilot of the new data collection strategies allowed them to adjust their strategy as they prepared to scale up. Important processes in developing the capacity to use data include adopting a system-level focus, collaborating on meaningful indicators of progress, understanding local circumstances and expertise, communicating regularly to share process and learning with relevant audiences, and balancing collective and organization-specific priorities.

Training and professional development to providers on using technology and data must be ongoing to mitigate the effects of high turnover and to further develop the skills of more experienced staff. Organizations also need dedicated staff—either internal system staff or external research partners—with skills in data analytics to effectively use data.