ASQ: A Guide to After-School Quality

ASQ: A Guide to After-School Quality

Learn the ins and outs of improving OST program quality in 5 easy steps with our team-based approach.

Are you just beginning to think about improving the quality of your OST program? Our ASQ guide is an excellent place to start. This 48-page document will walk you through our unique quality improvement process and teach you how to implement the 5-step model at your OST program site. As you follow the steps, you will lead your team in developing a common vision for your program and creating a roadmap to achieve that vision together.

Download a free 8-page sample ASQ Guide

Background:

The ASQ guide was originally developed in the mid-1990s and has changed over time to meet the needs of today’s OST programs. The first iteration was used as the basis for the National Afterschool Association program improvement and accreditation system. Today, the guide has the flexibility to support anyone interested in improving the quality of their program -- for self-assessment purposes, state level quality rating system advancement, or to become accredited from the Council on Accreditation -- and works alongside any data collection tools, questionnaires, or surveys.

Details:

Cost: $75 per OST program site

When you purchase the ASQ guide, you gain access to our online portal for three years where you can download sample agendas, letters, and tips to help you on the path to program improvement. Online portal access is renewable after the initial three years.

Frequently Asked Questions about the APAS System

What is the APAS System?
The APAS System is a flexible, research-based suite of tools to help programs measure and link their program quality to youth outcomes. It is made up of three tools: APT (to measure program quality), SAYO-S&T (to survey staff and teachers about youth outcomes), and SAYO-Y (to survey youth about their perspectives). The tools can be used alone or together and are not intended for individual youth assessment, nor as a diagnostic tool.

Who should use the APAS System?
The APAS System has been used by evaluators, out-of-school time directors, supervisors, curriculum and education coordinators, and 21st Century Community Learning Center personnel. These tools work best for programs who offer a variety of experiences for youth such as enrichment and leadership activities in addition to academic support.

Is the APAS System right for my program?
The APAS System works best for well-established programs (+1 year) that offer a variety of programming to 25 or more youth participants who attend on a regular basis. It is recommended for programs serving youth in Grades K-12 where youth attend at least 100 hours across a full school year or intensive session such as a day camp.

If your program does not fit the above description, we recommend that you start with only the APT, which is one tool in the APAS System, to measure the quality of your program and identify areas of strength and areas for improvement.

I am confused. How do I know if I need all of the APAS System or just one tool?
Each of the APAS tools can be used alone or in any combination. Tool selection should align with your program goals, the data you need, and your readiness to begin this work. For programs just starting data collection, select one tool that best aligns with what you need to know and your capacity to collect the data.

How much does the APAS System cost?
The APAS System is an excellent option for measuring program quality (with the APT tool) or youth outcomes (with the SAYO-Y or SAYO-S&T tools) at your program. The APAS tools are free and are provided upon completion of an in-person or online training program. Pricing for in-person training is determined on a case-by-case basis depending on your needs. Online training subscriptions for the APAS System are available for $600 per program site for one year, renewable at a reduced cost. Following successful completion of either in-person or online training, you can continue to use the tools at no cost as long as someone working at your site is trained in proper use and administration of the tools.

What do you mean by “program site”?
Pricing for the APAS System is determined by the number of program sites. A program site is defined as a physical address where programming for youth occurs. For example, if your organization offers three programs at two different physical locations, you have two program sites. Those with six or more program sites may be eligible for a group discount.

How many staff members can be trained to use the APAS System?
Training provides instruction on the proper use of the tools and customization. If you opt for an in-person training, we will work with you to determine how many staff members should be trained. If you purchase a 1-year online training subscription for your program site, you can train as many staff members who work at that site as you want.

Can I customize the tools in the APAS System?
Yes, through our in-person training or online training subscription, you will learn how to customize the APAS System tools to maintain the validity of your results.

Is the APAS System research-based?
Yes. The APAS System tools have been extensively piloted and scientifically tested beginning with the statewide Massachusetts Afterschool Research Study in 2004. Reliability and validity testing has continued with funding from the WT Grant Foundation and the Massachusetts Department of Elementary and Secondary Education.

Who collects and analyzes the data?
The method for data collection and analysis for the APAS System depends on the specific tool. For the APT tool, data collection and analysis can be done by the program site director or administrator, a designated team, or an internal or external evaluator. For the SAYO-Y or SAYO-S&T tools, there are two options: NIOST staff can manage data collection and analysis (for a fee) OR programs can collect their own data through Survey Monkey or another survey platform. If you choose to collect your own data, you will need to pay for a professional level license for Survey Monkey.

Frequently Asked Questions about SAYO-S&T or SAYO-Y

What is SAYO-S&T?
SAYO-S&T is a research-based measurement tool that uses staff and teacher surveys to collect data about specific youth outcomes that are aligned with your program goals. It can be used to measure any or all of 13 youth outcome areas that are scientifically linked to long-term positive youth development and academic and life success. SAYO-S&T is not intended for individual youth assessment, nor as a diagnostic tool.

How does SAYO-S&T work?
SAYO-S&T relies on pre- and post-participation surveys that are completed by OST staff (SAYO-S) and academic teachers (SAYO-T) to collect data about youth participants. Each outcome area is measured by asking staff and teachers to respond to questions related to observable youth behavior. It is designed to be used on its own or within the APAS System alongside APT and SAYO-Y.

What is SAYO-Y?
SAYO-Y is a research-based measurement tool that uses youth surveys to measure their experience, future planning, and sense of competence. Use SAYO-Y to survey your youth participants and collect data about what they are getting out of your program.

How does SAYO-Y work?
SAYO-Y is comprised of three brief pre- and post-participation surveys that are completed by youth participants to measure their program experiences, future expectations, and sense of competence. The surveys are designed for programs serving children from Grade 4 through Grade 12 and can be completed online. SAYO-Y is designed to be used on its own or within the APAS System alongside APT and SAYO-S&T.

Who should use SAYO-S&T or SAYO-Y?
SAYO-S&T and SAYO-Y have been used by evaluators, out-of-school time directors, supervisors, curriculum and education coordinators, and 21st Century Community Learning Center personnel to evaluate youth outcomes. These tools work best for programs who offer a variety of experiences for youth such as enrichment and leadership activities in addition to academic support.

Are SAYO-S&T or SAYO-Y right for my program?
The SAYO-S&T or SAYO-Y tools work best for well-established programs (+1 year) that offer a variety of programming to 25 or more youth participants who attend on a regular basis and receive at least 100 hours of programming across a full school year or intensive session such as a day camp. SAYO-S&T can be used for programs serving youth in Grades K-12, while the SAYO-Y is only available to youth in Grades 4-12.

If your program does not fit the above description, we recommend that you start with our APT tool to measure the quality of your program and identify areas of strength and areas for improvement.

How much does SAYO-S&T or SAYO-Y cost?
The SAYO-S&T or SAYO-Y tools are free and are provided upon completion of an in-person or online training program. Pricing for in-person training is determined on a case-by-case basis depending on your needs. Online training subscriptions for SAYO-S&T or SAYO-Y are each available for $200 per program site for one year, renewable at a reduced cost. Following successful completion of either in-person or online training, you can continue to use the tools at no cost as long as someone working at your site is trained in proper use and administration of the tools.

What do you mean by “program site”?
Pricing for SAYO-S&T or SAYO-Y is determined by the number of program sites. A program site is defined as a physical address where programming for youth occurs. For example, if your organization offers three programs at two different physical locations, you have two program sites. Those with six or more program sites may be eligible for a group discount.

How many staff members can be trained to use SAYO-S&T or SAYO-Y?
Training provides instruction on the proper use of the tools and customization. If you opt for an in-person training, we will work with you to determine how many staff members should be trained. If you purchase a 1-year online training subscription for your program site, you can train as many staff members who work at that site as you want.

Are SAYO-S&T or SAYO-Y research-based?
Yes. The SAYO-S&T and SAYO-Y are part of the APAS System, which has been extensively piloted and scientifically tested beginning with the statewide Massachusetts Afterschool Research Study in 2004. Reliability and validity testing has continued with funding from WT Grant Foundation and the Massachusetts Department of Elementary and Secondary Education.

Who collects and analyzes the data?
There are two options for data collection and analysis with the SAYO-Y or SAYO-S&T tools: NIOST staff can manage data collection and analysis OR programs can collect their own data through Survey Monkey. If you choose to collect your own data, you will need to pay for a professional level license for Survey Monkey.

Frequently Asked Questions about the APT

What is APT?
APT is a research-based tool to measure the quality of your program, identifying your areas of strength, and pinpointing areas for improvement. APT will give you a clear picture of your program by collecting data on practices that are linked to key youth outcomes: learning and skill building, program organization and structure, and supportive social environment. Tools are not intended for individual youth assessment, nor as a diagnostic tool or staff assessment tool.

How does it work?
APT relies on two data-collection methods -- the observation tool and the program questionnaire. The observation tool focuses on observable program practices while the program questionnaire, which is completed by you and your staff, examines “behind the scenes” aspects of program quality, like planning and school/parent connections. These two methods combined allow APT to look at the overall afterschool program, including homework time and activities.

APT is designed to be used either on its own or with either the SAYO-S&T or SAYO-Y tool, all three of which make up the APAS System.

Who should use APT?
APT has been used by evaluators, afterschool directors, supervisors, curriculum and education coordinators, and 21st Century Community Learning Center personnel. These tools work best for programs who offer a variety of experiences for youth such as enrichment and leadership activities in addition to academic support.

Is APT right for my program?
APT works best for well-established programs -- those that have been operating for one year or longer. APT is a great place to start if your program is new to assessment. For programs that have more advanced experience with assessment and/or data collection, we recommend the entire APAS System, of which APT is one of three included tools.

How much does APT cost?
The APT tool is free and provided upon completion of an in-person or online training program. Pricing for in-person training is determined on a case-by-case basis depending on your needs. Online training subscriptions for APT are available for $200 per program site for one year, renewable at a reduced cost. Following successful completion of either in-person or online training, you can continue to use the tools at no cost as long as someone working at your site is trained in proper use.

What do you mean by “program site”?
Pricing for APT is determined by the number of program sites. A program site is defined as a physical address where programming for youth occurs. For example, if your organization offers three programs at two different physical locations, you have two program sites. Those with six or more program sites may be eligible for a group discount.

How many staff members can be trained to use APT?
Training provides instruction on the proper use of the tools and customization. If you opt for an in-person training, we will work with you to determine how many staff members should be trained. If you purchase a 1-year online training subscription for your program site, you can train as many staff members who work at that site as you want.

Is APT research-based?
Yes. APT is part of the APAS System, which has been extensively piloted and scientifically tested beginning with the statewide Massachusetts Afterschool Research Study in 2004. Reliability and validity testing has continued with funding from the WT Grant Foundation and the Massachusetts Department of Elementary and Secondary Education.

Who collects and analyzes the data?
Data collection and analysis for APT can be done by the program site director or administrator, a designated team, or an internal or external evaluator.