It appears that 2,717 of the 3,649 students in the district took AR quizzes, or 74% of the total student population (Smith and Clark, p.53, 54). Those who averaged above 90% on the tests grew about 1.36 years in reading over the academic year, a substantial gain. But only 1010 students were in this group, 37% of those who took AR quizes. The rest, those scoring under 90%, averaged nearly exactly one year's growth in one year (mean = .97). AR thus appeared to be of benefit to about 1/3 of those who took the quizzes, about 28% of the students in the district.

Reports from the Renaissance Website

The Renaissance website (renlearn.com) contains a number of short reports all claiming success for AR, and all submitted by "independent researchers." Because these reports appear on a commercial website, promoting a product, one would expect these reports to be positive. Nevertheless, some problems are revealed when the reports examined in detail. I present a few of these reports: All of involve comparison to published norms.

Institute for Academic Excellence, Report 5: Craven County, North Carolina.

AR was installed district-wide in Craven County in 1991. As evidence for the efficacy of AR, it is noted that the percentage of "pass rates" of Craven students (grades 3-8 combined) on tests of reading is consistently above the state average and increases from 1993 to 1998 from 66 to 81 percent.

We cannot interpret this increase as due only to the impact of AR, because state-wide scores also increased. Such increases are typical after new tests are introduced (Linn, Graue, and Sanders, 1990), but we do not know the history of state-wide testing in North Carolina. We also do not know whether Craven scores were above the state average before AR was installed.

We would like to know if Craven's scores increased more than the state-wide average. They did, but the difference is small. In 1993, after two years of implementation, four percent more Craven students passed the state examination in reading than the state-wide average (Craven = 66%, state = 62%). In 1998, the difference was eight percent (Craven = 77%, state = 69%). By 1998, all Craven students had been exposed to AR, and older students had had five full years of AR. At best, the impact of AR was an increase of just a few percent more students passing the state exam: Craven students improved 11% from 1993-98, and the state-wide improvement was 7%.

Institute for Academic Excellence 7: The Buford Elementary School (Georgia)

Buford Elementary introduced AR in 1996. The scores presented in this report are for grades 1 through 4 (830 students). The 1995-96 scores can be considered the pretest.

Table 3: Increase in Reading Comprehension on the FCAT at Buford Elementary

From: Inst. Ac. Ex. 7
1995-96 = pretest

The gains presented in table 3 are impressive. According to the report, this represents reading growth of three years over the two year period.

A look at the state of Georgia website data on Buford revealed one potential flaw in this cheerful picture. Buford fifth graders in 2001 scored at the 50th percentile in reading. These students were in first grade in 1996-97 and in second grade in 1997-98, and should have participated in the AR program. If their scores at these times were typical of other students at Buford, they were scoring at the 64th percentile in 96-97 and the 70th in 98-99. What happened? Perhaps Buford dropped AR. If so, it means AR had no lasting effect. Another reason could be due to the high mobility rate at Buford: The report noted that Buford has a mobility rate of 25%. Perhaps many of these fifth graders never did AR.

4