Everyone notices when students mark the wrong answers on Illinois’ annual achievement tests. But this year, the grown-ups who run the schools made so many errors that much of the statewide data released Wednesday was seriously distorted.
A Tribune analysis found that numerous local school districts failed to properly classify students by race, economic status and language proficiency or reported the wrong enrollment numbers for students who took the state tests administered last spring.
For example, the state reported that 99.8 percent of the low-income children in Harvey School District 152 were not tested in reading–an impossibility given that nearly all of the district’s pupils are poor and 96.1 percent took the test.
The errors do not affect the overall scores reported Wednesday for districts and schools, which are accurate. Still, flaws in the information about demographic subgroups are not just a bookkeeping problem. They make it impossible to determine whether schools are artificially inflating their scores by not testing students in low-scoring groups.
These sorts of mistakes could also come back to haunt Illinois schools next year as they try to comply with the federal No Child Left Behind Act, which calls for schools to be punished if they don’t test 95 percent of children in various categories, including low-income, minority and special-education students.
Rigorous monitoring scarce
State and local education officials across the nation are scrambling to generate the kind of detailed data required by the new federal law. As the requirements become more complex, the potential for error rises–something officials say Illinois cannot afford given the high stakes involved.
Illinois relies on self-reporting by local districts, without a rigorous monitoring system to discover errors. Although the flawed data was in the hands of local school districts for more than a month for review, state and local school officials said the mistakes were not detected until the Tribune unearthed them.
“Clearly, there was an oversight and a lack of attention to detail on the local level,” said State Schools Supt. Robert Schiller, who said he would work with local districts to create a quality control system.
If test participation rates are below 95 percent next spring, schools that receive federal poverty money could face serious consequences, such as being forced to offer students tutoring services or the option to transfer and be bused to another school.
The problem with the data is especially apparent in south suburban Cook County, where the state reported that in a number of districts, more than 99 percent of low-income students were not tested. Yet the districts are composed almost entirely of poor children and most of those children were tested, local officials said in interviews.
If the numbers released by the state had been correct, those districts’ classrooms would have been virtually empty on test-taking day, so even a cursory review should have flagged that something was amiss, school officials acknowledged.
Besides Harvey, districts with incorrect test participation rates for low-income children in the Chicago area include Mt. Prospect District 57, Ford Heights School District 152 and Posen-Robbins Elementary District 143 1/2. But a Tribune analysis estimated that an additional 1,064 schools out of 3,915 statewide reporting test scores may not have properly labeled low-income students.
“Our district’s focus has been on raising achievement levels and not on the percentage of students taking the test, which has not been a problem for us,” said Lela Bridges, superintendent of Harvey District 152. “It was not on the radar to look at that figure closely and it got by me.”
Schools chief is shocked
In another kind of error, the state data shows that Midland Community Unit District 7 near Peoria had the state’s highest rate of children not taking the test. The state said almost half, or 44.5 percent, of Midland’s students took no reading test, a finding that shocked Supt. Kirby Rogers when he was contacted by the Tribune.
“I know that is wrong,” Rogers said.
He discovered that the district had reported an incorrect enrollment number for high school students taking the Prairie State Achievement Exam, which is given only in the 11th grade. Instead of the 65 11th graders eligible to be tested, Kirby said the high school’s total enrollment was entered, or 267 students.
As a result, the state concluded that only 65 of 267 eligible students had taken the test. That affected the participation rate for the entire district, which also includes elementary schools and a junior high.
Though the state said it was up to local districts to notice the errors, superintendents said the state must share responsibility for making sure data is accurate, especially in light of the new federal legislation.
Schiller said it is unlikely the state will recalculate and correct this year’s data. But he said the state may entertain appeals from districts if the incorrect numbers negatively affect them under the new federal law.
“The data now is almost eight months old. It would create a logistical nightmare to go back and recode it,” Schiller said. The focus, he said, now must be on making sure districts understand the importance of accurate student information.
Even with the reporting problems, the data suggests that hundreds of Illinois schools are in danger of not meeting federal requirements next year. High schools with large numbers of truant or special-education students may fall short of the federal requirement that each school test at least 95 percent of children in various categories.
And in some cases the numbers are sufficient to flag alarming trends. In Chicago, where officials acknowledge that not all special-education students have been tested in the past, the state data shows that 13.6 percent of those students did not take a reading test, compared with a little less than 1 percent for special-education students statewide.
The Tribune’s analysis did not detect widespread errors in Chicago’s reporting for special-education students. But even if the final number is lower than 13.6 percent, it would still be significantly higher than the state average.
Chicago Public Schools officials said they had not yet carefully reviewed the special-education data, but chief accountability officer Philip Hansen said Chicago has not emphasized testing for special-education students, or using those students’ scores to judge a school’s performance.
“You’re talking about changing a mind-set, and that will come with time,” said Hansen, a member of a statewide task force appointed by Schiller to examine assessment and accountability issues in Illinois.
U.S. provision stirs questions
The federal requirement that 95 percent of special-education students be tested is particularly problematic, many officials said.
The state has told districts that only a few special-education students should be permitted to take the Illinois Alternate Assessment, designed for severely disabled students. As a result, many special-education students are being forced to take the same state test as other children, even though it is often too difficult for them.
In one district, a school official reported that some students who took the state test drew pictures on the answer sheets instead of filling in the bubbles. Ultimately, those students were counted as not taking the state test, which then decreased the district’s percentage of special-education students tested.
“To make special-education students take inappropriate tests–that’s what’s happening, and it’s lunatic,” said Belleville School District 118 Supt. James Rosborg, who has criticized current special-education testing procedures as a member of the state assessment and accountability task force.
School systems where special-education students live in residential homes or treatment centers also are reporting problems. If those students do not take the state test, their lack of participation is tallied at their neighborhood school, even if they have never been enrolled there.
“Some of the residential centers refuse to let the students be tested,” Rosborg said. “A local district has no control over them, but it still counts against you.”
