Teacher quality under the microscope

Policy makers grapple with how to define successful teaching -- and whether financial incentives can help spur excellence

By Meris Stansbury, Associate Editor Thursday, August 20, 2009
[Posted on eSchoolNews Sunday, August 23, 2009 http://www.eschoolnews.com/news/top-news/index.cfm?i=60271 ]

It's no secret that one of the keys to creating better schools is to raise the quality of teaching in the nation's classrooms. But how to identify, and encourage, high-quality teaching is proving to be a challenge.

Several efforts to address this question are under way. The Bill & Melinda Gates Foundation has just launched a five-year, $500 million initiative to quantify what, exactly, makes a teacher effective and how to tie that to student achievement. And the Obama administration has cited improving teacher quality as one of four education-reform areas it plans to target in particular.

Education Secretary Arne Duncan has said he supports merit pay for teachers--a practice linking raises or bonuses to student achievement. He also said test scores alone should not decide a teacher's salary, "...but to somehow suggest we should not link student achievement to teacher effectiveness is like suggesting we judge sports teams without looking at the box score."

Duncan also is using federal stimulus dollars to press the issue.

Later this year, states will compete for a piece of $5 billion in "Race to the Top" stimulus funding, which rewards those states and school systems that adopt innovations the Obama administration supports. Whether officials tie student data to teacher evaluations will be a consideration in awarding the grants, said Duncan.

Although relatively rare, the use of pay-for-performance programs appears to be growing, albeit slowly. According to analyses of data from the "Schools and Staffing Survey" administered by the U.S. Department of Education, 13.6 percent of districts rewarded excellence in teaching in 1999-2000, and 14 percent rewarded excellence in teaching in 2003-04.

In 2003-04, 19.6 percent of districts said they rewarded some schools for excellence in teaching through a school-wide bonus or additional resources for a school-wide activity, and 15.4 percent of districts said they provide a cash bonus or additional resources to individual teachers to encourage effective teaching.

The key challenge in implementing pay-for-performance systems, experts agree, is how to define teacher excellence. The most obvious way would be to look at student achievement, as Duncan wants to do. But that's controversial, as many people believe test scores alone paint an unfair or incomplete picture of a teacher's contribution.

A recent survey, "Exploring the Possibility and Potential for Pay for Performance in America's Public Schools," conducted by the American Association of School Administrators (AASA), revealed the motivations and concerns that influence superintendents' consideration of pay-for-performance systems.

Out of 536 school administrators from 45 states, 45 percent expressed moderate to strong interest in pay-for-performance programs, and five percent of all respondents were already pursuing pay-for-performance programs for teachers in their districts.

The top three indicators school leaders would use in determining performance-based pay were student achievement (89 percent), teacher evaluations (68 percent), and teacher attendance (54 percent).

But defining student achievement should mean more than calculating test scores, many observers say.

"I don't believe merit pay based solely on test scores is appropriate," said Marc Liebman, superintendent of the Berryessa Union School District in San Jose, Calif. "The research I did [for] my doctorate ... indicated that using test scores as a hammer doesn't work. I don't think giving teachers more pay to get higher test scores will get the desired result, except in cases where teachers teach to the test, not to the students--which totally misses the point of high-quality instruction."

Education Sector, an education think tank, and public opinion research company FDR Group surveyed a national sample of teachers on their attitudes toward a variety of teacher policies, including compensation reforms. They found that fewer than half (42 percent) favored incentives for "teachers whose students routinely score higher than similar students on standardized tests."

In analyzing these results, as well as other survey data, the nonprofit Center for American Progress (CAP) determined that teachers are more likely to support programs that rely on a variety of measures of teacher performance, rather than those that rely on only one measure (such as test scores).

Also against merit pay based solely on student achievement scores is the Economic Policy Institute, which recently released a report titled "Teachers, Performance Pay, and Accountability: What Education Should Learn from Other Sectors."

The study, conducted by economics professors Scott J. Adams and John S. Heywood at the University of Wisconsin-Milwaukee, claims there are "significant downsides of reward-punishment systems based on quantitative outcomes, whether in the public or private sector."

"Contrary to the claims of advocates of teacher merit pay, relatively few private-sector workers have pay that varies in a direct formulaic way with their productivity--and the share of such workers is probably declining," Haywood states.

He adds, "Formulaic reward structures often reward only a few dimensions of productivity and run the risk of causing workers to abandon effort in the dimensions not rewarded."

Growth models and peer evaluations

If pay-for-performance programs should be based on multiple measures and not simple test scores, what should those measures be, and how can they be implemented properly?

According to CAP, programs should be designed to discourage teachers from "overly focusing on test-taking strategies, or repetitious drill on a narrow band of curricular material believed to be heavily represented on state exams."

CAP suggests that more inclusive approaches can be derived from high-quality evaluation systems that incorporate a variety of student outcomes and observations of teacher performance, in addition to value-added estimates. Also, the group suggests that evaluation systems should be correlated with student achievement data--teachers whom observers rate more highly should garner better results in terms of student achievement.

Joe Kitchens, superintendent of the Western Heights Public Schools in Oklahoma City, Okla., believes test scores must be a basic consideration, but there should be other types of measures as well, such as "other types of engagement with students that would matter greatly."

Based on Kitchens' observations within his district, he thinks student achievement depends on a number of factors, some of which are outside the control of teachers--such as student mobility. "We have pretty strong evidence ... that we must become more effective in dealing with mobile families, and any positive solution regarding merit pay would have to take the issue of mobility into consideration," he said.

Districts that are experimenting with pay-for-performance systems include the Plano, Texas, Independent School District, which has come up with an innovative "growth model" to help analyze how much of a student's achievement can be attributed to his or her teacher--and how much can be attributed to other factors. Twenty-eight of Plano's 68 schools are participating in this state-funded program as part of the Texas Educator Excellence Awards, or TEEG.

"We developed a measure for use, [called] the ‘Plano Effect Score,' that provides an accurate look at student achievement over the course of a school year using multiple measures and not simply the state assessment," said Jim Hirsch, associate superintendent for technology at Plano ISD. "Using a growth measure based on individual starting points eliminates the reliance on other factors. Our research shows that a minimum of 70 percent of students' growth is determined by their starting point; other factors are minimal if the analytics are modeled correctly."

Plano uses the program to target its schools that are most in need of intervention, and teachers at the selected campuses who teach the core subjects of reading, English, math, and science are eligible to receive a minimum award of $1,400 based on the academic growth of the students instructed by their grade-level or departmental team.

Other examples of pay-for-performance models that include many measures of success include Arkansas' Achievement Challenge Project, Denver's Pro Comp Program, North Carolina's Mission Possible Program, and Vanderbilt University's Teacher Advancement Program.

Higher salaries for high-need subjects

Along with merit pay, Duncan also has touted the idea of providing incentives for high-quality teachers to teach in underserved schools, as well as rewarding teachers in math, science, and other "high-need" fields that remain in their subjects for a number of years.

The National Center on Performance Incentives has studied the impact of TEEG--a pay-for-performance program in Texas that targets high-poverty, high-performing districts--and concluded that awards of $3,000, on average, reduced the predicted turnover rate to less than a quarter of the rate that was expected before the program was introduced.

Another estimate comes from SUNY-Albany researcher Donald Boyd, using data from New York City schools. He reported it would be necessary to "pay teachers an additional $2,900 to induce them to teach in a classroom with a 25-percentage-point increase in the proportion of minority students, but only an additional $350 to teach in a classroom with a 25-percentage-point increase in the proportion of students receiving free or reduced lunch."

A charter school in New York City's Washington Heights, called the Equity Project, will open in September, paying teachers $125,000 a year--twice as much as the average New York City public school teacher earns, and about two and half times as much as the national average for teacher salaries.

The school marks the biggest experiment yet as to whether the promise of higher pay can lead to better teaching. Teachers at the school also will be eligible for bonuses based on school-wide performance, of up to $25,000 in the second year.

The school's leaders believe that talented teachers, not technology or smaller class sizes, create success. Teachers so far include an accomplished violinist who uses neuroscience lessons during her music classes, two teachers with Ivy League degrees, and a phys ed teacher who used to work as Kobe Bryant's personal trainer.

The Equity Project will open with 120 fifth graders chosen this past spring in a lottery that gave preference to children from the neighborhood and to low academic performers. It will grow to 580 children in grades five though eight, with 28 teachers.

The school will use public money for everything but its building. Teachers will be responsible for duties usually given to assistant principals (there are none), and other positions which will not be filled. There will be no deans, substitute teachers, or teacher coaches. Teachers will work longer hours and more days, and they each will have around 30 students. They will not have the same retirement benefits as members of the city's teacher's union; they also can be fired at will.

"Teacher efficacy is the single most effective component [to] increased student achievement, outside of the students taking ownership of their own learning," said Hirsch. "However, extremely high salaries might help recruit teachers with high efficacy, but it won't guarantee it."
"Studies nationally have shown that, statistically [speaking], salary is not correlated with student success or achievement," said Liebman, "so I am not sure whether this idea has merit or not."
According to CAP, additional evidence on the effectiveness of merit pay programs will be available beginning in 2011 from the National Center on Performance Incentives, which received a five-year, $10 million grant from the U.S. Department of Education's Institute of Education Sciences to study the effectiveness of performance incentives.

One study will employ a randomized experimental design to assess the causal impact of a pilot program in Nashville public schools. The program allows math teachers to earn bonuses of up to $15,000 per year, conditional on their students' gains on state exams.

"Successful implementation of pay-for-performance models will require an ongoing dialog with all members of the education community to arrive at a solution that best serves the nation's students," said Randy Collins, AASA president and superintendent in Waterford, Conn.

"At this point, there are more questions than answers in the research on performance pay," said CAP, "but existing research findings suggest that the strategy holds promise for improving student achievement. There is less information about the impact on teacher recruitment and retention."
For its part, the American Federation of Teachers (AFT)--the nation's second largest teachers union, behind the National Educational Association (NEA)--acknowledges the traditional single-salary schedule for teachers has shortcomings. The group "believes it is time to explore viable, fair, and educationally sound teacher compensation options that will raise salaries while contributing to efforts already under way to ensure high-quality, well-prepared teachers for all students."

The AFT is encouraging its local unions to explore various teacher compensation systems based on their local conditions, though it does not support merit-pay systems based on individual test scores. According to the AFT, a professional teacher compensation system could include financial incentives to teachers who acquire additional knowledge and skills, or who agree to teach in low-performing or hard-to-staff schools. Such compensation proposals also could include increased pay for school-wide improvement, mentoring new and veteran teachers, and teaching in high-need areas, the group said.

The NEA did not respond to interview requests before press time.