Posted on 31-12-2002

More Schools Rely on Tests, but Study Raises Doubts
By Greg Winter, NYT, 28 Dec 2002

Rigorous testing that decides whether students graduate, teachers win
bonuses and schools are shuttered, an approach already in place in more
than half the nation, does little to improve achievement and may actually
worsen academic performance and dropout rates, according to the largest
study ever on the issue.
Advertisement

With calls for accountability in public education mounting, such
make-or-break exams have become cornerstones in at least 28 states in the
drive to improve public schools. The idea is that by tying test scores to
great consequences, the learning process will be taken that much more
seriously and tangible progress will be all the more likely. The approach
is also central to some of President Bush's sweeping education overhaul,
lending even greater momentum to the movement known as "high stakes"
testing. But the study, performed by researchers at Arizona State
University and financed by teachers' unions that have expressed skepticism
about such tests, found that while students show consistent improvement on
these state exams, the opposite is typically true of their performance on
other, independent measures of academic achievement. For example, after
adopting such exams, twice as many states slipped against the national
average on the SAT and the ACT as gained on it. The same held true for
elementary-school math scores on the National Assessment of Educational
Progress, an exam overseen by the United States Department of Education.

Trends on Advanced Placement tests were also worse than the national
average in 57 percent of those states, while movement in elementary-school
reading scores was evenly split — better than the national average in half
the states, worse in the other half. The only category in which most of the
states gained ground was middle-school math, with 63 percent of them
bettering the national trend. "Teachers are focusing so intently on the
high-stakes tests that they are neglecting other things that are ultimately
more important," said Audrey Amrein, the study's lead author, who says she
supported high-stakes tests before conducting her research. "In theory,
high-stakes tests should work, because they advance the notions of high
standards and accountability. But students are being trained so narrowly
because of it, they are having a hard time branching out and understanding
general problem-solving."

The study was commissioned by the Great Lakes Center for Education Research
and Practice, a Midwestern group of six state affiliates of the National
Education Association, which has opposed using any one test to determine
when students graduate, schools get more money and teachers are replaced.
The research is sure to be a subject of fierce debate among educators, and
its methodology has already drawn some criticism, though an independent
panel of researchers at other universities has concluded that the findings
are valid.

Perhaps most controversial, the study found that once states tie
standardized tests to graduation, fewer students tend to get diplomas.
After adopting such mandatory exit exams, twice as many states had a
graduation rate that fell faster than the national average as those with a
rate that fell slower. Not surprisingly, then, dropout rates worsened in 62
percent of the states, relative to the national average, while enrollment
of young people in programs offering equivalency diplomas climbed. The
reason for this is not solely that struggling students grow frustrated and
ultimately quit, the study concluded. In an echo of the findings of other
researchers, the authors asserted that administrators, held responsible for
raising tests scores at a school or in an entire district, occasionally
pressure failing students to drop out.

In lawsuits, educators have testified that students were held back rather
than promoted to a grade in which high-stakes tests were administered, and
that others were expelled en masse shortly before testing days. But neither
those witnesses nor this study has been able to quantify that circumstance
nationally, or prove that it has substantially influenced dropout rates.
Educators have long complained that the threat of serious consequences
means that teachers focus on little else, sometimes building their lesson
plans entirely around the contents of the test. That would not necessarily
be a problem if the state exams were based on a comprehensive curriculum,
said Eva Baker, co-director of the National Center for Research on
Evaluation, Standards and Student Testing at U.C.L.A. But as often as not,
the state exams are given in the absence of such a framework, leaving
teachers to fill in the gaps on their own, sometimes with an overzealous
reliance on test-taking drills. "The most perverse problem with high-stakes
tests," Ms. Baker said, "is that they have become a substitute for the
curriculum instead of simply a measure of it."

Some researchers suggested that the study might have actually understated
the consequences of high-stakes tests, particularly for dropout rates,
because it relied on government statistics. "Officially reported dropout
statistics are pretty suspect in a lot of places," said Walter Haney, a
professor at the Lynch School of Education at Boston College, pointing out
that students who leave school to get a G.E.D. are not always counted as
dropouts. "The real results are probably worse."

A larger question raised by the study is what effect, if any, it will have
on the public debate over high-stakes testing. While many educators will
most likely hold it up as proof that such exams are flawed, largely because
they appear to offer inadvertent encouragement to schools to constrain the
curriculum and squeeze out underachievers, others see the issue as more
open-ended.