Collecting and Using Data
Data is a powerful tool for cities to use in identifying gaps in service, telling a community’s story and improving quality. Afterschool and community leaders rely on data to measure the scope and impact of afterschool and summer programs in their city. Data is also critical to drive a system of quality improvement and to allocate scarce resources more effectively.
Data collection, analysis and use can be extremely challenging for small and rural communities due to a lack of resources and staff capacity. Communities like Mapleton, Ore., with a population of roughly 800, have only one afterschool program that serves their entire K-12 population. With one program, communities do not have the resources, staff capacity or need to have a sophisticated data system. Oftentimes, small communities leverage the data system and staff capacity of the school district to provide critical information. Some smaller cities, such as Northfield, Minn., have a number of different afterschool providers and have grown their capacity and resources to invest in their own data system.
The Role of Municipalities
Municipal leaders can play a key role in ensuring data is not only collected, but analyzed and used in decision-making. This is key when resources are scarce and communities need to efficiently allocate limited resources.
The first step is understanding the city’s afterschool landscape. To collect information about programs already in place, mayors and city councilmembers can employ several useful approaches. For example, communities can conduct a survey of local providers and use the data to generate baseline information about the participants and programs, such as the demographics of each participant and the types of activities in which they are engaged. Questions like, “What age is being served?” and “Where are program(s) located?” can help cities understand what resources they do have, who is being served, where the program(s) are located and what potential gaps may exist.
Cities can also conduct a youth mapping exercise where young people are surveyed to better understand their needs and the challenges and barriers that may prohibit them from accessing afterschool opportunities. All of this data is critical to making informed decisions that are in the best interest of young people.
Many of the small cities featured in this publication have used data to make the case for increased afterschool opportunities. For example, the data gathered by the City of Juneau, Alaska identified all of the different afterschool providers in the city, who they served, where they were located and what activities they offered. Through this effort Juneau found that the entire middle school population was underserved in the out-of-school time hours. As a result, their efforts have focused on increasing availability of programs targeted to middle school students.
Similar to Juneau, city leaders in Hammond, La. , knew there was a shortage of programming in their city, but realized they needed to better understand the afterschool landscape before they created a plan. As a result, they conducted a number of different surveys, including one of the provider community, a youth survey and a survey of parents. This information is being used to drive the development of the Hammond Youth Education Alliance’s strategic plan.
Collection and Use of Data
Small communities that are recipients of 21st Century Community Learning Center funds — the only federal funding stream that supports afterschool — are able to leverage the Profile and Performance Information Collection System (PPICS).
According to the U.S. Department of Education, PPICS is “designed to collect comprehensive information on program characteristics, services and performance data across a broad range of outcomes from state-administered 21st CCLC programs.” This system provides annual reports to providers regarding their performance the previous year. While this data is only collected on an annual basis, the system provides an infrastructure and capacity for small programs to collect important data, including participant demographics and attendance.
Research has shown that tracking attendance is the single most important data point for afterschool programs. The more often a student participates in afterschool programs and the longer they persist over time, the greater the impact on a range of academic, health and social and behavioral outcomes.
In order to track this information more effectively and efficiently, many cities are investing in their own management information systems. This is particularly true for large cities, but smaller communities are increasingly investing in this technology as well. Cities like Northfield, Minn., have invested in data systems to help them track attendance and measure program impact.
Measuring Program and Youth Outcomes
The PRIMEtime Collaborative in Northfield, Minn., utilizes a breadth of data and data gathering methods to analyze the impact and efficacy of the afterschool programs in its network. Cityspan and the Survey of Academic and Youth Outcomes (SAYO) are two methods PRIMEtime has used to collected data.
Cityspan, which was piloted for four months before being formally used as a data tracking tool, collects data on attendance across sites (for both its afterschool and college access programs), activity information and out-of-school time/summer programs, among other things. PRIMEtime administers the SAYO survey annually to evaluate youth experiences in their afterschool programs.
PRIMEtime is intentional about respecting the privacy of its participants, and therefore refrains from using personal data such as test scores and grades. Instead, the network focuses on collecting activity, demographic and attendance data. Information on attendance, for example, has helped PRIMEtime develop a common definition of attendance for their programs: a student who makes twenty or more visits to any PRIMEtime programming is defined as a regular attendee. Of those students who attended programs regularly, almost 93 percent reported an improved grade point average and/or improved standardized test scores from the previous year.
The PRIMEtime network offers introductory training opportunities to providers in an effort to equip them with knowledge about the data tools, and has at least one staff member dedicated to analyzing the information. However, data collection is not mandatory.
Data has been critical for the PRIMEtime Collaborative. It has helped tell a powerful story about the importance of afterschool programming in the community.