Tag: universities

The peer-review fetish: Let’s abolish the gold standard!

I’ve just been reading two books on the climate-change debate, both focusing on the so-called “hockey stick graph”: Michael Mann’s The Hockey Stick and the Climate Wars: Dispatches from the Front Lines, and A. W. Montford’s The Hockey Stick Illusion: Climategate and the Corruption of Science. I’ll comment on these in a later post, but right now I want to comment on the totemic role that the strange ritual of anonymous peer review plays for the gatekeepers of science.

One commonly hears that anonymous peer review (henceforth APR) is the “gold standard” for scientific papers. Now, this is a reasonable description, in that the gold standard was a system that long outlived its usefulness, constraining growth and innovation by attempting to measure something that is inherently fluid and abstract by an arbitrary concrete criterion, and persisting through the vested interests of a few and deficient imagination of the many.

That’s not usually what people mean, though.

An article is submitted to a journal. An editor has read it and decided to include it. It appears in print. What does APR add to this? It means that the editor also solicited the opinion of at least one other person (the “referee(s)”). That’s it. The opinion may have been expressed in three lines or less. She may have ignored the opinion.

Furthermore, to drain away any incentive for the referee(s) to be conscientious about their work,

  • They are unpaid.
  • They are anonymous. We know how well that works for raising the tone of blog comments.
  • Anonymity implies: Their contributions will never be acknowledged. If they contribute important insights to the paper, they may be recognised in the acknowledgement section: “We are grateful for the helpful suggestions of an anonymous referee.” Very occasionally an author will suggest, through the editor, that a referee who has made important contributions be invited to join the paper as a co-author. More commonly, a paper will be sent from journal to journal, collecting useful suggestions until it has actually become worth publishing.*
  • No one will ever take issue with any positive remarks the referee makes, as no one but the authors (and the editor) will ever see them. Negative comments, on the other hand, may get pushback from the author, and thus need to be justified, requiring far more work.
  • Normally, the author will be forced to demonstrate that she has taken the referee’s criticism to heart, no matter how petty or subjective. This encourages the referee to adopt an Olympian stance, passing judgement on what by rights ought to be the author’s prerogative.

Of course, I don’t mean to say that most referees most of the time don’t do a very conscientious job. I take refereeing seriously, and make a good-faith effort to be fair, judicious, and helpful. But I’m sure that I’m not the only one who feels that the incentives are pushing in other directions, and to the extent that I do a careful job, it is mainly out of some abstract sense of duty. I am particularly irritated when I find myself forced to put original insights into my report, to explain why the paper is deficient. I would much rather the paper be published as is, and then I could make my criticism publicly, and then, if I’m right, be recognised for my contribution. Continue reading “The peer-review fetish: Let’s abolish the gold standard!”

The Long Room: An academic allegory

Richard Tames’s A Traveller’s History of Oxford describes the “Long Room” of New College, a range of first-floor latrines built over a huge cesspit. Robert Plot, first superintendent of the Ashmolean Museum rhapsodised in the late 17th century that it was

stupendous… so large and deep that it has never been emptied since the foundation of the College, which was above three hundred years since, nor is it ever likely to want it.

The book also notes that this historical appraisal was in fact erroneous, as the pit had in fact, according to College records, been emptied in 1485.

The author does not make clear whether this description is intended as an allegory of academic productivity.

What’s the Matter with Economics?

One of the most politically important economics results of recent years has been the paper by Reinhart and Rogoff on the link between high sovereign debt and low GDP growth. This work is something I’d been following for a while, as R&R’s book was one that I’d admired greatly. Their work claimed to show a strong negative correlation between sovereign debt/GDP ratio and ensuing GDP growth, and was reported as saying that 90% debt/GDP ratio marks a cliff that an economy falls off, killing future growth. This was seized upon by proponents of austerity as proof that budget cuts can’t wait.

As reported here and here by Paul Krugman, and here and here by Matt Yglesias, it now turns out that the result isn’t just theoretically misguided, it’s bogus. Economists who struggled to reproduce the results finally isolated a whole raft of errors and dubious hidden assumptions that completely undermine the conclusion. Only the most blatantly ridiculous fault was an error in their Excel spreadsheet formula that caused them to exclude important sections of the data from their computation. You’d think that this couldn’t get any worse, but instead of apologising abjectly, R&R have tried to argue that none of this was really essential to their real point, whatever that was.

My main thoughts:

  1. Do economists really do their analysis with Excel? I find this kind of shocking, like if I found out that some surgeons like to make their incisions with flint knives, or if airline pilots were calculating their flightpaths with slide rules. Once you accept that premise, it’s not surprising that they made a blunder like this. I’m not a snob about technology. Spreadsheets are great for doing payrolls, and for getting a look at tables of numbers, and doing some quick calculations. But they’re so opaque, they’re not appropriate to academic work, and they’re so inflexible that it’s inconceivable to me that someone who analyses data on a more or less regular basis would choose to use them. Continue reading “What’s the Matter with Economics?”

Universities and charity

 

Here’s a weird, but hardly novel, controversy: Charity tax row: Oxbridge joins revolt.

The Oxford and Cambridge vice-chancellors wrote privately to Chancellor George Osborne saying his plans risked undermining the culture of university philanthropy. UK universities, which raised some £560m from charitable gifts last year, want him to rethink. Ministers want to stop tax avoidance. Mr Osborne says he is shocked by thescale of legal tax avoidance by multi-millionaires. Under current rules, higher-rate taxpayers can donate unlimited amounts of money to charity and offset it against their tax bill to effectively bring the amount of tax they pay down, sometimes to zero. But from 2013, uncapped tax reliefs – including those on charitable donations – are to be capped at £50,000 or 25% of a person’s income, whichever is higher… An Oxford University spokeswoman said that the government’s own policy emphasised the role of private and philanthropic investment, rather than the public purse. “A step that penalises the government’s own approach seems ill-considered.”

Hmmm. How about this alternative statement:

The university’s own justification depends on its promoting self-consistent argument, rather than specious self-serving sophistry. “An argument that contradicts the university’s own raison d’etre seems ill-considered.”

Continue reading “Universities and charity”

Where the money is…

Mathematical finance as an accessory to crime

Not long after I finished my PhD in probability theory, a significant fraction of the field was devoured by the financial mathematics moloch. Particularly in Europe, probability theory positions disappeared, to be replaced by openings in financial mathematics, which either went unfilled or cycled among a very few senior researchers and a few quick-change opportunists (and, gradually, their fledgeling academic progeny).

Everyone felt they had to get in on the action, and of course there was a certain amount of positive feedback. When many jobs chase few graduates, it generates huge demand among students for training in such a demonstrably burgeoning field. Obviously, the academic feedback was limited by the fact that most of the eager young ‘uns were seeking employment in banks, not in academia — but the banks were hiring as well. Anyway, just about 10 years ago, a Dutch colleague asked me if I might be interested in joining his own institute’s planned financial mathematics group, for which they were proposing to create TEN new positions. My reply was that finance did not interest me as a topic of research, but I added that there was something unseemly — bordering on unethical — in mathematicians’ headlong chase after banking lucre. The current generation of mathematicians is the trustee of a vast and powerful system of analysis, whose creators were supported, honoured, and financed by public institutions. What is it but a crime, when we abscond with the fruits of this scholarship, and sell it off (cheaply) to banks, who will use it to extract billions of dollars from financial markets? Continue reading “Where the money is…”