Harvard Professor Caroline Hoxby Discusses School Accountability at Abt Associates 2004 Annual Meeting

Dr. Caroline M. Hoxby, professor of economics at Harvard University, described the impact researchers are making in improving elementary and secondary education in a lecture at the Abt Associates 2004 annual stockholder meeting in July.  Dr. Hoxby presented the Company's twelfth annual Walter R. Stellwagen lecture.

Because of the wealth of data now available through school accountability, Dr. Hoxby believes that researchers have the opportunity to begin to make the link between input-such as teachers, curriculum, and textbooks-and student outcomes.  Armed with results and analyses from several recent studies, Dr. Hoxby presented an overview of school accountability nationwide and the opportunities that presents for researchers to examine longstanding puzzles and challenge traditional assumptions.  While accountability is not new, said Dr. Hoxby, the Elementary and Secondary Education Act — known as "No Child Left Behind" — has further reinforced the steps many states have already taken in that direction.

Elements of school accountability that are of particular interest to researchers include:

1. Statewide testing in reading and math in grades 3 through 11
2. Mandatory reporting of test results at the school level with details for various subgroups based on race, income, disability, and English proficiency
3. Databases that contain longitudinal information on students, teachers, and schools

"Whether or not you like accountability as a parent, or a voter, or just a citizen of the United States, as a researcher you almost have to like accountability," said Dr. Hoxby.  "I really think accountability is here to stay."

According to Professor Hoxby, who is also the director of the Economics of Education Program for the National Bureau of Economic Research, a distinguished visiting fellow at the Hoover Institution and a member of the Koret Task Force on K-12 Education, legislators like accountability because it is relatively inexpensive-the "most expensive" programs in the United States generally cost less than one quarter of 1 percent of per pupil spending, and most of these are only as costly as they are because a state is in the "expensive" and temporary phase of developing its own comprehensive tests.  The business community favors accountability because employers want better educated, more literate, more numerate employees.  They think that their tax dollars should be spent wisely to produce such potential employees, said Dr. Hoxby.  Also, employers are currently paying for a lot of training themselves that is really make-up for employees' poor basic reading, writing, and math skills.  Overall, she said, accountability gets strong public support.  From researchers' points of view, accountability helps overcome the lack of outcome data that has been a chronic problem for those studying educational issues.

Why Accountability?

The push for accountability, Dr. Hoxby explained, comes from the government's need to know how tax dollars are being spent.  Overall, state and federal governments pay about 58 percent of the cost of educating students.  Local governments contribute about 42 percent of the cost.

"You can not expect the state and federal governments to spend a lot of money and not find out what's happening," she said.

Dr. Hoxby noted that the average per pupil expenditure nearly doubled in the 30-year period from 1970 to 2000, rising from just below $5,000 a year to $9,500 per pupil.  Over the same period, student achievement remained virtually flat, with only modest improvements in reading and math among eighth-graders.  That is not the result legislators would expect from such an increase in educational spending, Dr. Hoxby said.

One defense, she said, could be that states have been pumping the extra dollars into programs for disadvantaged students.  But the outcome for those students challenges that theory.  Dr. Hoxby noted that the achievement gap between average and disadvantaged students also remained quite flat, with little sign of improvement.

"We're the highest spenders and, embarrassingly, we're not the highest achievers," said Dr. Hoxby, comparing the United States with other countries.  "In general, there is a feeling we need to do something more.  That's one of the reasons why accountability is here to stay."

Dr. Hoxby said one of her objectives as an education economist is to encourage policy-makers to think about the data now available to them, and to use it to help create new reforms.  For example, by tracking the performance of students over time it is possible to determine which of their teachers were the most effective.  Using that data, schools could then consider programs that reward teachers for their individual or group performance, said Dr. Hoxby.  Or, in districts that employ teachers with alternative certifications, schools can use the outcome data of students to better track the performance of those teachers, she added.

"Many policies that have been unsophisticated in the past can become more sophisticated if we make full use of the data," said Dr. Hoxby.  "One of the things I think we need to do as researchers is help give states and federal government guidance on how they can actually use data better.  We're the experts in that.  They're not."

Applying Accountability Data

Dr. Hoxby offered three examples of the kinds of questions educational researchers have explored using the new data.

Charter Schools:  In one study by Hoxby & Rockoff (2004), researchers examined a charter school system in Chicago, the largest of its kind in the United States.  The question they were probing, said Dr. Hoxby, was whether or not students do better in charter schools.  The charter schools they studied in Chicago were almost all in historically rough neighborhoods, the student bodies are largely black and Hispanic, and the schools hold lotteries to determine which students to admit.  Researchers found that when students entered a charter school in the earlier grades-kindergarten through third-grade-they had substantial gains in their reading and math scores.  Those results, however, were not repeated for students who enrolled in charter schools between sixth and eighth grade.  Their gains were very small and not statistically significant.

Busing:  In another case, Dr. Hoxby described research by Samms (2004) that examined whether black students have higher achievement rates when they are bused to and enroll in white schools.  Researchers looked at data from the Charlotte-Mecklenburg school district in North Carolina for insight.  They studied the effect on test scores when the percent of black students in a school increased by 10 percent because of busing.  The result?  The change in reading and math scores was barely discernable.  That finding could suggest that, contrary to what people generally think, the racial composition of a school's student body is less important to academic achievement than the school's staff, resources, neighborhood, and other attributes that do not change when busing changes.

Teachers:  In a third example, Dr. Hoxby shared the findings of a study by Rockoff (2004) about whether teachers affect student achievement.  "Everyone thinks that teachers matter," she said, "but it's been very hard to show that their qualifications-in particular their teacher credentials, whether or not they have a master's degree, their experience-matter." In this case, the accountability data allowed the researchers to isolate the systematic value-added of teachers, said Dr. Hoxby.

Researchers found that with some teachers, almost no students showed gains in reading and math.  With other teachers, however, almost every student improved in those subjects, said Dr. Hoxby, and the teachers repeated that affect on students year after year.  The results suggest that there are many differences among teachers, and that not all of teachers are approaching the material in the same way, said Dr. Hoxby.  Evidence like this might lead school districts to consider different pay structures or incentive programs for teachers, she added.

One example Dr. Hoxby cited is a reward system tried in Florida.  There, teachers in Advanced Placement (AP) classes earned a bonus for every student who scored at a certain level on the AP test.  "Teaching AP classes became a lot more desirable.  Teachers went back and got different training," said Dr. Hoxby.  "You can often start off with simple individual rewards and once the data starts flowing in, people's behavior actually starts changing."

Attracting Effective Teachers

Pay may well play a roll in the declining aptitude level of teachers in the United States, said Dr. Hoxby, citing research by Hoxby & Leigh (2004).  Pay may also affect which schools teachers decide to work for, she added.  Increasingly, women are able to earn amounts comparable with men in professions other than teaching.  And both men and women can earn more outside of teaching.  In 1963, 20 percent of women in the 95th percentile of aptitude became teachers, said Dr. Hoxby, but by 2000, only about 1 percent of women in that high aptitude range chose teaching as a career.

"Being a high aptitude person in teaching now in the United States is extremely rare," said Dr. Hoxby.  In addition, teachers have experienced a dramatically reduced pay differential for aptitude.  Dr. Hoxby pointed out that in 1963, there was a 37 percent pay differential between teachers with the highest aptitudes and those in the lowest quarter of aptitude; today, that pay difference is less than 5 percent.

With pay differentials between districts so slight, research by Loeb et al. (2002) and Hanushek et al. (2002) is showing that quality of life concerns such as shorter commutes and easy-to-educate students are becoming deciding factors when teachers consider which schools to work in.  But there is at least one way to attract and keep high caliber teachers in tougher environments, Dr. Hoxby added, and charter schools are showing the world how.

"Extra sensitivity of pay does seem to be working for charter schools because they are getting teachers who are slightly more likely to be high in aptitude and have more science courses and math courses," she said.  "If you're willing to pay enough more, they will come and teach in difficult schools."

However, the idea of pay differentials doesn't sit well with teacher unions.  Concerned about favoritism, unions have opposed the concept for years, said Dr. Hoxby.  But just as the new data from school accountability may help guide policy-makers, so may it impress the unions.

"An interesting question will be what the unions do once we start to have more and more objective data about the differences in teacher performance," Dr. Hoxby said.