Test Schemes Look More Like Lotto Than Lessons

The new state educational accountability system is starting to look more like a game show than a thoughtful educational reform strategy.

In October, the state will announce $550 million in rewards to schools for improving student achievement scores. Then, around November, a tiny percentage of Californian teachers in schools that made the greatest gains each will receive additional $25,000 bonuses. The problem with all these rewards is that they are based on test scores in a few subjects, not the breadth of content covered by the state standards. Furthermore, the big cash windfalls for a few schools send the wrong messages--that we value huge one-year gains more than long-term continuous improvement--to all schools. In addition, the test scores on which the accountability index is based are not all that accurate, so winning a big bonus may involve considerable chance.

There are three types of incentives that will be awarded this fall, all of which have some shortcomings:

* The Governor’s Performance Award program will distribute $227 million to schools that meet 5% achievement growth targets overall and promote comparable growth for low-income and minority students. Schools will earn $150 per student, with local parent and teacher groups deciding how the money is used.

* The School Site Employee Performance Bonus program will award an additional one-time bonus to all staff in these same schools. It is estimated that the $350 million earmarked for this program will result in awards of about $1,200 per person.

* The Certificated Staff Performance Incentive Act is the most troubling piece of the system. These rewards will go to a few schools that show the greatest gain in student scores above their targets. Only schools in the bottom half are eligible to receive rewards, and the money will be distributed in huge chunks until $100 million is spent. One thousand teachers in the schools achieving the greatest test score gains will receive $25,000 each. This represents about one-third of 1% of the 300,000 teachers in California. Another 3,750 teachers will receive $10,000 each, and 7,500 teachers will receive $5,000 each.

Incentives like these may be effective if they target the right outcomes and encourage the right behaviors, but this set of tests and rewards has shortcomings on both fronts. For example, the system rewards performance in reading and mathematics in elementary schools but not in science, social studies, art or music.

The $25,000 bonuses send even more troublesome signals. Imagine what would happen in any organization if a tiny fraction of people could earn $25,000 by scoring best on some specific measure of performance. People would find every means possible to look good on that measure. Some co-workers would try harder to do the job they are supposed to do. But some would focus their energies only on those things that pay and ignore the rest of their responsibilities. Worst of all, some would resort to tricks or deceit to improve their scores.

There is ample evidence from other states that teachers react in all these ways when they are faced with high-stakes accountability systems. For example, they shift class time to the subjects that are tested and take it away from those that are not. Teachers also spend excessive time in test preparation activities of questionable educational value. When the same test is used year after year, it is tempting to teach the specific test questions rather than the broader skills they are supposed to measure.

Will these negative events happen in California? There’s no guarantee, but they have occurred in every state that has been studied, and California’s $25,000 teacher rewards are the highest in the nation.

What should be done?

* Eliminate these lottery-style payouts to a handful of teachers and focus on long-term sustained improvement. The state’s underperforming schools program, which provides expert guidance and financial support for schools over a three-year period, is a far better approach to improvement than these gigantic bonuses.

* Broaden the base of information to be tested on so that performance in science, social studies, music and art matters.

* Change the test from year to year to lessen the temptation to teach specific questions to boost scores.

* Institute an audit testing mechanism using different tests with a sample of students to provide an independent check on the degree to which scores are compromised by inappropriate test preparation and shifts in curriculum.

The accountability system may help our schools improve, but not unless these problems are addressed.