This week, economists have been astonished to
find that a famous academic paper often used to make the case for
austerity cuts contains major errors. Another surprise is that the
mistakes, by two eminent Harvard professors, were spotted by a student
doing his homework.
It's 4 January 2010, the Marriott Hotel
in Atlanta. At the annual meeting of the American Economic Association,
Professor Carmen Reinhart and the former chief economist of the
International Monetary Fund, Ken Rogoff, are presenting a research paper
called Growth in a Time of Debt.
At a time of economic crisis, their finding resonates -
economic growth slows dramatically when the size of a country's debt
rises above 90% of Gross Domestic Product, the overall size of the
economy.
Word about this paper spread. Policymakers wanted to know more.
And so did student Thomas Herndon. His professors at the
University of Massachusetts Amherst had set his graduate class an
assignment - pick an economics paper and see if you can replicate the
results. It's a good exercise for aspiring researchers.
Thomas chose Growth in a Time of Debt. It was getting a lot
of attention, but intuitively, he says, he was dubious about its
findings.
Some key figures tackling the global recession
found this paper a useful addition to the debate at the heart of which
is this key question: is it best to let debt increase in the hope of
stimulating economic growth to get out of the slump, or is it better to
cut spending and raise taxes aggressively to get public debt under
control?
EU commissioner Olli Rehn and influential US Republican
politician Paul Ryan have both quoted a 90% debt-to-GDP limit to support
their austerity strategies.
But while US politicians were arguing over whether to inject
more stimulus into the economy, the euro was creaking under the strain
of forced austerity, and a new coalition government in the UK was
promising to raise taxes and cut spending to get the economy under
control - Thomas Herndon's homework assignment wasn't going well.
No matter how he tried, he just couldn't replicate Reinhart and Rogoff's results.
"My heart sank," he says. "I thought I had
likely made a gross error. Because I'm a student the odds were I'd made
the mistake, not the well-known Harvard professors."
His professors were also sure he must be doing something wrong.
"I remember I had a meeting with my professor, Michael Ash,
where he basically said, 'Come on, Tom, this isn't too hard - you just
gotta go sort this out.'"
So Herndon checked his work, and checked again.
By the end of the semester, when he still hadn't cracked the puzzle, his supervisors realised something was up.
"We had this puzzle that we were unable to replicate the
results that Reinhart-Rogoff published," Prof Ash, says. "And that
really got under our skin. That was really a mystery for us."
So Ash and his colleague Prof Robert Pollin encouraged
Herndon to continue the project and to write to the Harvard professors.
After some correspondence, Reinhart and Rogoff provided Thomas with the
actual working spreadsheet they'd used to obtain their results.
"Everyone says seeing is believing, but I almost didn't believe my eyes," he says.
Thomas called his girlfriend over to check his eyes weren't deceiving him
But no, he was correct - he'd spotted a basic
error in the spreadsheet. The Harvard professors had accidentally only
included 15 of the 20 countries under analysis in their key calculation
(of average GDP growth in countries with high public debt).
Australia, Austria, Belgium, Canada and Denmark were missing.
Oops.
Herndon and his professors found other issues with Growth in a
Time of Debt, which had an even bigger impact on the famous result. The
first was the fact that for some countries, some data was missing
altogether.
Reinhart and Rogoff say that they were assembling the data
series bit by bit, and at the time they presented the paper for the
American Economic Association conference, good quality data on post-war
Canada, Australia and New Zealand simply weren't available.
Nevertheless, the omission made a substantial difference.
Thomas and his supervisors also didn't like the way that
Reinhart and Rogoff averaged their data. They say one bad year for a
small country like New Zealand, was blown out of proportion because it
was given the same weight as, for example, the UK's nearly 20 years with
high public debt.
"New Zealand's single year, 1951, at -8%
growth is held up with the same weight as Britain's nearly 20 years in
the high public debt category at 2.5% growth," Michael Ash says.
"I think that's a mistaken way to examine these data."
There's no black and white here, because there are also
downsides to the obvious alternatives. But still, it's controversial and
it, too, made a big difference.
All these results were published by Thomas Herndon and his
professors on 15 April, as a draft working paper. They find that high
levels of debt are still correlated with lower growth - but the most
spectacular results from the Reinhart and Rogoff paper disappear. High
debt is correlated with somewhat lower growth, but the relationship is
much gentler and there are lots of exceptions to the rule.
Greece is an example of a country with high debt that has suffered a slump
Reinhart and Rogoff weren't available to be interviewed, but they did provide the BBC with a statement.
In it, they said: "We are grateful to Herndon et al. for the
careful attention to our original Growth in a Time of Debt AER paper and
for pointing out an important correction to Figure 2 of that paper.
It is sobering that such an error slipped into one of our papers despite
our best efforts to be consistently careful. We will redouble our
efforts to avoid such errors in the future. We do not, however, believe
this regrettable slip affects in any significant way the central
message of the paper or that in our subsequent work."
Accidents do happen, and science progresses through the
identification of previous mistakes. But was this a particularly
expensive mistake?
"I don't think jobs were destroyed because of this but it
provides an intellectual rationalisation for things that affect how
people think about the world," says Daniel Hamermesh, professor of
economics at Royal Holloway, University of London.
"And how people think about the world, especially politicians, eventually affects how the world works."
Discovering a spreadsheet error was never going to end the
debate over austerity - and nor should it, according to Megan McArdle,
special correspondent for Newsweek and The Daily Beast.
"There is other research showing that you can have these
slowdowns when you get to high levels of debt," she says. "We have a
very vivid [example] in Greece."
Thomas Herndon 's view is that austerity policies are
counter-productive. But right now he's delighted that the first academic
paper he's ever published has made such a splash.
"I feel really honoured to have made a contribution to the policy discussion," he says.
No comments:
Post a Comment