Excerpted from the SEP by the BELL program, a subgrantee of the Edna McConnell Clark Foundation.
Alignment of each outcome with the logic model presented earlier in the SEP is described here.
How each variable from the logic model will be measured is detailed here.
For each measure, whether the measure will be developed or has already been developed (i.e., a commercially available, off-the-shelf measure) is explained here. (Please scroll down to view all of the highlighted content.)
The outcome measures for the impact analysis will closely track the logic model presented earlier in this plan and are shown explicitly in Exhibit 3. Within the initial three-year stage of the evaluation, we will focus on the early outcomes of improvements in basic skills over the summer (with an end-of-summer test) and in the school year following program participation (with a spring test). To the extent feasible, we will utilize existing state or local tests of reading and math already being fielded in participating schools. This lessens the testing burden on students and schools and provides a measure most relevant for practitioners and policy makers. If we use existing tests, test scores from different tests across sites will be standardized (z-scored) by district to make it possible to pool impacts from different local assessments.
In most cases, for the random assignment design, we anticipate we will need to field a special achievement test for the evaluation for the end-of-summer achievement measure. Existing tests will typically be fielded in the spring of each academic year for students in grades three and up in reading, math, and other academic subjects. For example, in the first district to begin the evaluation, Springfield, MA, the Massachusetts Comprehensive Assessment System testing will provide spring scores on English language arts, math, and science/technology for students in grades three and up. In this district, the GRADE tests of math and reading tests will be fielded to sample members near the end of the BELL program to provide an end-of-summer measure of reading and math achievements. The GRADE test is a widely used assessment with a long track record of use and scores can be provided as scaled scores, normal curve equivalents, and percentiles.
If we are able to conduct a regression discontinuity analysis of BELL’s impacts in Detroit, we will rely on the fall administration of the Michigan state test for an end of summer measure of reading and math achievement. For longer follow up, we will either rely on a spring test fielded by Detroit Public Schools (if an appropriate one exists) or use the fall administration of the Michigan test in the subsequent year.
We will also field a survey of students in our random assignment impact analysis to collect information on students’ attitudes toward school and learning and academic self-concept, and increased involvement in the community. We are currently exploring the advantages of various scales for use in our student surveys. Past studies of summer programs have used the Perceptions of Ability Scale Scores (PASS) survey and this is under consideration for this study, but it has not proven sensitive enough to pick up small changes in attitudes. We are also examining other survey measures for use in the survey. At this point, we have not budgeted for a parent survey given the substantial incremental costs of such a survey. If we are able to conduct a parent survey, we will also be able to estimate impacts on social skills at home and the engagement of parents. Even without a survey, we will explore these topics through interviews with summer program staff.
Unavailable:
Unavailable:
Unavailable:
Unavailable:
There is no content in this SEP example that identifies with the currently selected SEP Guidance Checklist element.
