The only mantra more predictable than budget woes for the Chicago Public Schools is hand-wringing over test scores.
As summer ebbs and debate begins anew over how to find money to open schools on time, educators and board members are saying that budget cuts must not come from any part of the educational program that will affect test scores.
But a close examination of what really happened the last time test scores rose-and then plummeted-in Chicago suggests that anyone who wants to actually improve learning will have to do more than change the numbers.
Test scores had been creeping up steadily through the 1980s. Then from 1989 to 1990, the bottom fell out. It appeared that the grade school children in the system had gotten dumber-a lot dumber-overnight.
But all the system had done was change the test. As a result, the number of 3rd graders who scored above average in reading fell from four out of 10-a respectable number for a large urban district-to two out of 10, a virtual educational meltdown.
In 3rd-grade math, the drop was almost as bad, from five out of 10 to three out of 10. Likewise, the number of 6th graders who scored above average in reading and math on the Iowa tests fell by a third, and similar declines were seen in 8th grade.
The declines were too large to have happened by chance in a district with more than 300,000 children in 475 elementary schools.
And they were too sudden to be caused by a drop in the intelligence of children, the teaching skills of teachers, the management style of principals or the expectations of parents.
Instead, what appeared to be a sharp decline in achievement was actually more like resetting a fast clock closer to the correct time by dropping an old, out-of-date test and starting fresh with a new one.
Chicago’s Iowa test scores had become artificially high during the 1980s due to a process that has been dubbed the “Lake Wobegon effect” by educational testing experts. It is named after author Garrison Keillor’s mythical town in Minnesota where “all the women are beautiful, all the men are strong and all the children are above average.”
In the Lake Wobegon effect, teachers’ familiarity with an old test, combined with the out-of-date standards used to develop the old test, artificially boost test scores to the point that nearly all children score above the average.
“If you use the same test for seven years, the teacher knows what is there,” said Carole Perlman, director of testing for Chicago. “The children would get higher scores, but they couldn’t use the information in new situations.”
The fast clock robbed Chicago parents, teachers, principals and taxpayers of a clear benchmark with which to measure pupil progress. It also misled school officials and politicians into thinking things were better-and then with the new test, worse-than they really were.
But reaction to that misleading decline continues to squander time and resources on raising test scores, rather than focusing on developing a pupil’s love of reading and a confidence in his ability to learn, many teachers say.
“We may have created a nation of test takers, but not a nation of good students,” said Maureen Connolly, a 21-year veteran teacher at Healy Elementary School. “Good test takers are too narrow. We beat them to death with knowledge, and we don’t look at whether they can use that knowledge.”
Perlman said she recommended that the system buy a new test in 1990 because “testing had become very, very high stakes in Chicago . . . and we were very concerned about getting valid measurements.”
Children did worse on the new test because it was harder and their teachers were unfamiliar with it.
The old 1978 Iowa test had become so familiar to the teachers, Perlman said, that even without any intention to cheat, they were coaching their pupils for specific questions.
For example, if a teacher knew that the test always had math problems involving perimeters or discounts, she would be likely to emphasize those types of problems in her class-possibly using the same examples as she recalled from the test.
If she knew there was a reading paragraph about the life cycle of the carrot, she might use that in a science class rather than other vegetables and, with no intention to cheat, ensure that the children had a head start on the test.
When Chicago’s children were given the unfamiliar test, school after school dropped out of the “above average” range, sometimes dramatically so.
In some schools, the percentage of pupils who scored above average in reading or math fell by 30, 40 or 50 points.
Some slippage was to be expected. Because of the newer norms, pupils had to do better to get the same score they had received on the 1978 test.
But the drop that could be attributed to the “renorming” should have been only about 10 to 20 points, depending upon the grade and skill tested, according to H.D. Hoover, professor at the University of Iowa and head of the Iowa Tests of Basic Skills.
Chicago used different forms of the 1990 test in 1991 and 1992, and will use different tests every year from now on, Perlman said. The state-required tests that produce the annual “school report cards” give more consistent results and are not prone to these fluctuations.
Crucial decisions are still tied to standardized test scores in Chicago.
They are used in part to determine if children are promoted or held back. One 3rd-grade teacher said she had 12 of her pupils retained the year the scores dropped, a much larger number than usual.
The 7th-grade test scores are used as one admission criterion by Chicago’s selective public and private high schools.
Teachers find this particularly damaging to children who are bright and creative but who “do not score well.”
And improvement in test scores is part of the contracts of some principals under the school reform law, which allows local school councils to hire and fire principals.
Hoover called “nearly every one (of Chicago’s uses of the Iowa tests) negative. We write in our manuals that this is not how to use the test. But standardized tests are convenient tools . . . used frequently by policymakers as a way to appear to be doing something about education without spending much money.”
Hoover said the primary use of the Iowa tests is to inform parents and teachers about the academic strengths and weaknesses of pupils and classrooms.
Teachers say they object to the tests because poor results are self-perpetuating and undue emphasis on test robs time from more productive teaching.
“By the time they get to the upper grades, they have been getting low scores for so many years that, to them, a test equals something that they do not know,” said Jeraldyne Saines, 7th- and 8th-grade language arts teacher at Bethune Elementary School.
“The tests drive the curriculum,” said Elizabeth Burns, a 3rd-grade teacher at Bethune. “You have so much pressure to answer test questions that you aren’t going to think about doing a science experiment or having the children write a book.”
Yet as much as she criticizes the tests, Saines admits that she works test-taking skills into lessons.
“If they are going to change the format even a little on the tests, I have to know so that I can work it into my teaching,” said Saines, who has been teaching for 22 years. “When I am teaching something in language arts, I tell them, `These are the kinds of information they want you to know on your Iowas.’ “
