Perry and Abecedarian Projects FAQ
The Perry Preschool Project and The Carolina Abecedarian Project are two widely acclaimed early childhood education intervention programs that have followed participants for over half a century, reporting promising results along the way. Critics, however, have grown skeptical about the accuracy and relevance of these programs, especially as these programs have increasingly been used to justify larger-scale early childhood efforts. Below are responses to common concerns.
"The sample sizes in Perry and Abecedarian were too small for results to be conclusive."
We can have confidence in the Perry and Abecedarian results even if the studies had relatively small sample sizes because this limitation is taken into account within the measure of statistical significance. Results from Perry and Abecedarian are so strong that they survive conservative statistical procedures that account for small sample inference. Furthermore, when using a 5% significance level, one would expect that 5 out of 100 hypotheses would be “significant” just by chance. Heckman’s widely accepted analysis of Perry outcomes shows that significant differences between treated and untreated children remain even after accounting for the possibility that some outcomes may appear significant due to chance alone.
“The positive effects on IQ scores initially seen in Perry and Abecedarian disappeared over time.”
It is true that both Perry and Abecedarian did not show any positive IQ effects just a few years following the intervention program. Decades of follow-ups, however, have shown extremely encouraging results along dimensions such as schooling, earnings, crime involvement, and health. Professor Heckman and colleagues believe that we may be focusing on the wrong set of traits, as the “soft stills” gained by students who have attended these programs—conscientiousness, self-control, motivation, etc.— tend to be much more predictive of success in adulthood. It is these skills that facilitate the better performance of treatment students on achievement tests despite performing no better on IQ tests. Another possible explanation for the “fade-out” in IQ, offered by Perry researchers, is that these results many have more to do with the way in which we test for improvement than the amount of improvement itself.
“Perry and Abecedarian were high quality programs. Thus, findings from these programs cannot be generalized to large-scale attempts at offering pre-kindergarten.”
There is no doubt that the quality of the program is of supreme importance to its success. The Perry Preschool Project, for example, hired teachers with bachelor’s degrees and certification in education, maintained at least a 1 to 6 staff-to-child ratio, and included weekly teacher home visits. That said, Heckman and colleagues are confident that state, federal and private early childhood education programs that model programs like Perry can deliver a 7-10% annual return on investment, which is comparable to the post World War II return on equity in the US stock. This projected return serves as a strong economic incentive to have high standards for large-scale programs as well.
"Head Start has failed to replicate the results seen in Perry and Abecedarian. According to the Head Start Impact Study (HSIS), the first randomized controlled trial of the program, all positive effects faded by 3rd grade.”
While results have been disheartening, HSIS has a host of limitations that must be considered. To begin with, it compares apples to apples: Over 60% of control children either attended alternate preschool programs or were accepted into Head Start through a later lottery while 18% of students in the treatment group chose not to attend Head Start at all. These factors dilute any gains achieved by Head Start. Another limitation is that the study cannot control for the different amounts of attention children receive when they enter elementary school. Since control children were behind at the end of the program, it is possible that teachers worked to catch them up. But even in the highly unlikely case that these factors do not affect reported results, it is still possible that we will observe positive effects emerge at later years. The truly remarkable impacts of Perry and Abecedarian were not seen until much later in the lives of participants. Also, while HSIS is in its early stages, there are numerous long-term, quasi-experimental studies that find Head Start children to attend more years of schooling, earn higher incomes, live healthier, and engage less in criminal behavior. Considering this, it is especially important that we see HSIS through before condemning Head Start.
Head Start FAQ
"What do the results from HSIS determine about Head Start?"
According to the Head Start Impact Study (HSIS), the first randomized controlled trial of the program, all positive effects faded by 3rd grade. While results have been disheartening, HSIS has a host of limitations that must be considered. To begin with, it compares apples to apples: Over 60% of control children either attended alternate preschool programs or were accepted into Head Start through a later lottery while 18% of students in the treatment group chose not to attend Head Start at all. These factors dilute any gains achieved by Head Start. Another limitation is that the study cannot control for the different amounts of attention children receive when they enter elementary school. Since control children were behind at the end of the program, it is possible that teachers worked to catch them up. But even in the highly unlikely case that these factors do not affect reported results, it is still possible that we will observe positive effects emerge at later years. The truly remarkable impacts of Perry and Abecedarian were not seen until much later in the lives of participants. Also, while HSIS is in its early stages, there are numerous long-term, quasi-experimental studies that find Head Start children to attend more years of schooling, earn higher incomes, live healthier, and engage less in criminal behavior. Considering this, it is especially important that we see HSIS through before condemning Head Start.
"Are there any studies besides HSIS that can tell us about Head Start?"
Despite the emphasis placed on HSIS, there are in fact many long-term, creative, quasi-experimental Head Start studies we can gain insight from. Similar to Perry and Abecedarian, these studies find Head Start children to attend more years of schooling, earn higher incomes, live healthier, and engage less in criminal behavior. One often-cited study from the National Bureau of Economic Research used the Panel Study of Income Dynamics survey (PSID). The PSID has been administered to 18,000 individuals (4,802 households) since 1968 and the 1995 questionnaire happened to ask whether individuals attended Head Start. Based on this, researchers found 4,000 adults to focus in on. This group was composed of participants of Head Start and a control group of siblings who did not attend Head Start. (Garces, 2000) Researchers chose to use siblings in order to limit variations in environment between the control and treatment groups (even HSIS does not do this). What researchers could not address was that (1) children were non-randomly selected into Head Start and (2) there could have been recall error by the participants when filling out the PSID. The first point is largely true with HSIS as well and the second point seems contrived as it is hard to imagine a substantial number of participants simply forgetting that they attended (or did not attend) Head Start. Thus, it is well worth giving results from this study weight in our discussion: Head Start participants were 9% more likely to complete high school. Among white participants, this number was closer to 30%. White participants were also 28% more likely to attend college than siblings who attended no preschool and 20% more likely than those who attended an alternate preschool. African American participants also showed greater educational attainment and were 12% less likely to have been involved with criminal activity that their siblings. (Garces, 2000) Overall, measured results were substantial and bode well for HSIS. Another noteworthy study conducted by Ludwig and Miller used a different but equally creative approach to overcome selection bias. They looked to the discontinuity in Head Start funding across counties to find effects of the program at the county level. The reason there was a discontinuity in funding is that the Office of Economic Opportunity provided grant-writing assistance for the 300 poorest counties in 1960, resulting in 50-100% higher funding rates in counties that barely made the 300 compared with counties right above that threshold. This small difference was smoothed before the start of the program, meaning these two groups of counties were at the same poverty levels but had very different access to Head Start. Ludwig and Miller found evidence for a half-year increase in schooling attainment and 15% increase in likelihood of attending college within the Head Start heavy counties. They even found a substantial decrease in mortality in these counties. (Ludwig, 2008) Overall, researchers aggregating these non-experimental but still rigorous studies suggest that benefits do outweigh the costs of Head Start at a ratio as large as 7-to-1. (Ludwig, 2008) While it is reasonable to argue that these studies cannot be given the same weight as a randomized, controlled trial, it would be foolish to ignore them completely. At the very least they should serve as a reason to continue HSIS. Continuing would not be a blind walk but rather a walk looking to confirm steps already taken.
"Do we have any reasons to believe that positive effects can re-emerge at later years?"
But even in the highly unlikely case that these limitations do not hinder the reported results from perfectly reflecting reality, it is still possible that we will observe positive effects emerge at later years of HSIS. The hope for this comes from observing the impressive outcomes —despite initial fadeout— seen in small-scale, long-term studies such as the Perry Preschool Project and the Abecedarian Project. Perry Preschool was a randomized, controlled high-quality program run in the early 1960s. Children who won a coin toss were given 2.5 hours of teaching each weekday morning for two years (Schweinhart, 2005). These children, along with a control group, have now been followed for almost 50 years. Analyzing questionnaires and records, researchers found that participants were less likely to be arrested, earned substantially higher incomes and attended an average of almost 1 full year more of school (Schweinhart, 2005). They needed an average of 1.3 years fewer of special education services and were 26% less likely to receive government assistance (Social Programs That Work). The Carolina Abecedarian Project was also conducted in the 60s and was even more intensive, involving a center-based child care service from birth to age 5 for 8 hours a day (Currie, 2001). Similar to Perry, researchers found a host of positive long-run effects. One particularly striking result was that, at age 21, treatment children were twice as likely to be attending a four-year college (Currie, 2001). When referring to Perry Preschool, Abecedarian, and other studies like it, people tend to focus their attention on these extremely promising long-term results. It is forgotten that many of these studies saw achievement test scores fall greatly in magnitude right after the intervention. In fact, both Perry Preschool and Abecedarian did not show any IQ results after just a year following the program. When questioned, Perry researchers suggested that this may have more to do with the way in which we test for improvement than the amount of improvement itself (Schweinhart, 2005). Regardless of the reason for this fade-out, it is important to consider that the same studies that are famously successful and fuel the excitement surround ECE were once in the same boat that HSIS is in now. Therefore, we should refrain from jumping to conclusions and instead be patient until we gain long-term observations from HSIS as well.
Rate of Return FAQ
"How is the return per $1 invested in human capital interventions by age of intervention calculated?"
The internal rate of return of a program has the exact same definition for any program to be evaluated. It is obtained based on calculating the benefits of the program net of their costs.
"How do you calculate benefits?"
By benefits we mean the difference between many variables of interest between the treatment and control groups of any program, i.e. people who received the program and people who did not. Examples of these variables are lifetime income, expenditure in health, welfare transfers, etc.
"How do you calculate costs?"
The calculation is analogous for the costs. Importantly, even when they do not enter into the program they do incur in some costs which are accounted for. An example of a cost for the control group would be the alternative preschool that mother take the children to when not admitted into the program.
"Were the measures of return similar to those you tracked in your analysis of the Perry Preschool Program?"
The evidence suggests that the earlier the program, the greater the rate of return. For the case of Perry, we calculated that the internal rate of return of the program is around 7 to 12 percent.