Don't trust Ed Trust

Don't trust Ed Trust. Substance 27 (6): 3. 2002.

Stephen Krashen

The powerful impact of poverty on literacy development has been well documented. Children of poverty, in addition to the obvious problems they face, have very little access to reading material ; they have fewer books in the home, inferior public libraries, inferior school libraries,and inferior classroom libraries, (e.g. Duke, 2000; Neuman and Celano, 2001). This means, of course, that they have fewer opportunities to read, and therefore make less progress in developing literacy.

The recent report from Educational Trust West (Ali and Jerald, 2001) appears, at first glance, to show that a significant number of children in poverty have overcome this problem. The report claimed to find 3,592 schools in the US that were "high-performing-high poverty" schools. In California alone, there were 355 high-performing-high poverty school. This result was considered sufficient to "dispel the myth" about the relationship between poverty and educational achievement, and was followed by newspaper articles proclaiming that these high-scoring schools can "offer a lesson" (New York Times, December 17, 2001; Los Angeles Daily News, December 16, 2001).

The Ed Trust Report deserves another look. It has serious flaws, and, in fact, shows exactly the opposite of what it says it shows.

Very few schools qualify. The number of schools classified as high-poverty high-scoring represents about 4% of the nation and state school population. Moreover, a closer look shrinks even this number to considerably. In fact, it shrinks it to nearly zero.

It is easy to qualify as high-scoring. A high-performing school was defined as one in which students in ANY grade scored in the upper third of the schools in its own state in EITHER math or reading. Thus, a good performance by one grade level (in some schools only one classroom) on one test can qualify a school as "high performing."

Consider the case of California. Of the 355 "high-scoring" schools in California, only 134 were high-scoring in reading. There are 8761 schools in California. This means that about 1.5% qualify as "high-flying schools." Of these 134, 83 managed to qualify because of children in only one grade level! This could be due to the performance of a few students in one classroom, perhaps even those from higher-income families (see below). We are now down to 51 schools, about half of one percent.

Scores can be based on students NOT considered high poverty. Ed Trust may claim that a grade in a high poverty school reached the upper 1/3, but not all the children at that grade level were high poverty. Consider the case of fourth graders at the Language Academy, a (magnet) school in San Diego. Academy fourth graders scored in the upper 1/3 of the state in reading, averaging 61. But the subset of economically disadvantaged children (n = 27) scored 42, while the advantaged children (n = 36) averaged 73. Fourth graders at Language Academy were classified as high scoring high poverty not because of the scores of its disadvantaged children but because of the scores of its advantaged children. Ed Trust does not present this kind of a breakdown of scores.

Ed Trust used a low standard for classification as "high poverty." A high-poverty school was defined as one in which at least 50% of the students were from low-income families. The California average is 46%.

The report has numerous inaccuracies. For California, several schools listed as high-poverty were not, and in many cases grade levels Ed Trust said were high scoring were not. The alternative analysis below presents details, as well as confirming that the number of "high-poverty high-scoring schools" is very very small.