Of hockey sticks and statistics

[Updated at bottom] I recently read two books on climate science — or rather, two books on the controversies around climate science. One book was Michael Mann’s book The Hockey Stick and the Climate Wars; the other The Hockey Stick Illusion by Andrew Montford.

Now, I am, by inclination and tribal allegiance, of the party of Michael Mann, one of the world’s leading climate scientists. He and his colleagues have been subject to beastly treatment by political opponents, some of which is detailed in his book. And I only picked up the Montford book out of a sense of obligation to see what the opposing side was saying. And yet…

Montford’s book makes a pretty persuasive case. Not that climate science is bunk, or a conspiracy, or that anthropogenic global warming is a fiction — there is far too much converging evidence from different fields to plausibly make that claim (and indeed, Montford never makes such a claim) — but that a combination of egotism and back-scratching has seriously slowed down the process of evaluation and correction of sometimes sloppy statistical procedures, and tarnished the reputation of the scientific community generally.

I admit to a certain bias here: The attacks on Mann’s work that Montford describes are statistical in nature, and Mann’s response reminds me of the tone that is all too common when statisticians raise questions about published scientific work. Montford has a remarkable amount of technical detail — so much that I found myself wondering who his intended audience is — and the critiques he describes (mainly due to the Canadian mining engineer Steve McIntyre) seem eminently sensible. In the end, I don’t think they panned out, put they were genuine shortcomings in the early work, and McIntyre seems to have done the right things in demonstrating the failure of a statistical method, at least in principle, and to have earned for his trouble only incomprehension and abuse from Mann and his colleagues.

In particular, like many empirical scientists, Mann seems not to have understood — or not to have wanted to understand — the importance of making it possible for people wanting to check the methodology to reproduce exactly the calculations that led to the reported results, by making data and code available. Without that, the best anyone can do who is suspicious of the method is try to redo the calculations with new code; but if the results come out differently, you have no good way to establish where the error is, making a formal criticism and correction extremely challenging and time-consuming.

Montford reports that Mann refused to share the computer code with McIntyre, saying that to do so

would be giving in to the intimidation tactics that these people are engaged in.

When I read this in Montford’s book I thought, this can’t be true. This sounds like a libellous invention. But then, Mann repeated the comment in his own book, and seemed rather proud of it. His argument for why he wouldn’t release his source code, in full, was

(1) Our source code wasn’t necessary to reproduce and verify our findings. [Other scientists…] had independently implemented our algorithm without access to our source code. (2) Our source code was our intellectual property. [NSF], which had funded our work, had already established that we had more than met the standards of disclosure of data and methods expected of NSF-funded scientists and that the specific source code we had written was our intellectual property. (3) While I was happy to provide source code — and had — to colleagues (including competitors) who were engaged in good faith attempts to assess our methods, important precedents were at issue here. Did we really want to head down the slippery slope of releasing proprietary materials indiscriminately? What other vexatious demands might be made of us and others?

Was it Samuel Johnson who said that intellectual property is the last refuge of a scoundrel. It’s hard to see why this would be considered a vexatious demand at all by someone with nothing to hide. Now, it could be that what he had to hide was that the code was actually a mess. But since he had already shared it with some others, it shouldn’t have been too difficult to whip it into shape to share with McIntyre.

Professor Mann’s comments remind me of the remark made by historian John Lewis Gaddis in a 1983 article on “The Emerging Post-Revisionist Synthesis on the Origins of the Cold War”:

A decade ago this subject was capable of eliciting torrents of impassioned prose, of inducing normally placid professors to behave like gladiators at scholarly meetings, of provoking calls for the suppression of unpopular points of view, threats of lawsuits, and, most shocking of all, the checking of footnotes.

Gaddis remark was made tongue-in-cheek, of course, but Michael Mann would presumably have argued, “Do we want to head down the slippery slope of allowing people to check our footnotes? What will they want to check next.

Another palaeoclimatologist, Phil Jones, managed to top Mann’s chutzpah by replying to a request for the data from a publication by writing

We have 25 years invested in the work. Why should I make the data available to you, when your aim is to find something wrong with it?

Mann’s book has some technical defences of his work, and some of them are reasonably convincing, but his method is mostly ad hominem: His opponents are bad people, arguing in bad faith. “Their intent wasn’t so much to contribute to the scientific conversation as it was to influence the public discourse.” Everyone who criticises him is shown to have not quite the right credentials (and/or biased funding sources). They publish on web sites rather than in peer-reviewed journals. But when they do publish in peer-reviewed journals, they’re not the right journals, and peer review isn’t that useful anyway. (Something I agree with, see here.) He quotes another scientist saying of his critics McIntyre and McKitrick, “As far as I know they’re quacks.” (Rhetorical tip: Playground insults don’t become cogent arguments as soon as you quote someone else making them.)

On the other hand, Mann gets the vapours when McIntyre “titled a post about a highly respected NASA climate scientist with the rhetorical question ‘Is Gavin Schmidt Honest?'” Throughout his book, Mann alternates snide aspersions and ad hominem attacks on his opponents, while expressing shock at the much milder critical tone that comes from the global-warming sceptic camp. One particularly silly passage is his dismay at an economist Hu McCulloch, who alleged that “he could not reproduce a tropical ice core record produced by Lonnie Thompson and colleagues, implicitly claiming either ineptitude or, worse, malfeasance.” Is that really what someone is claiming when he fails to reproduce the results? If Thompson is anything like Mann in his attitude toward helping other researchers to reproduce his work, I can think of a lot of other reasons why he might have failed, and decided he just needed to let other researchers sort it out.

Although Montford’s book preceded Mann’s, Mann seems to take no cognizance of its claims and arguments. In one particularly telling non-exchange, Montford describes, with detailed correspondence, how their technical comment on Mann’s original hockey-stick paper was rejected, after several rounds of refereeing, “principally because the discussion cannot be condensed into our 500-word/1-figure format”. Mann reports that the comment was rejected “for lacking merit”, but provides no evidence, simply dismissing the claim that “lack of space” was an issue by citing Nature‘s published policy which does not treat length as a primary criterion. Indeed. I would say this rather strengthens Montford’s argument, that the rejection was, at least, suspicious.

Update 31-1-2014: The paragraphs below referred to a lawsuit that Michael Mann has brought against a blogger. Commenter Rob Honeycutt has pointed out that I misinterpreted the news reports, which prominently referred to the blogger comparing Mann’s abuse of data to a pedophile’s abuse of children. The libel suit is, apparently, based on the allegations of fraud in the article.

I should also cite prominently the critical book review of Montfort on the climate change science blog realscience.org. I certainly don’t mean to praise Montfort’s book, which is excessively tendentious, to the point of paranoia at times. I was just dismayed that Mann’s book, with the weight of scientific respectability on its side, isn’t any better.

Interestingly, Professor Mann has filed a libel suit, but not over Montford’s accusation that he has been, at least, uncooperative and high-handed with his critics, and at worst has worked behind the scenes to manipulate peer review of work in climate science. Rather, he has sued a journalist for  comparing his abuse of data to a pedophile’s abuse of children. So his failure to sue Montford cannot be attributed to any hesitancy to hoist the weapon of lawsuit in an intellectual battle, and must therefore be taken as tacit confirmation that the scandalous statements and at least the broad outlines of the behaviour attributed to him in the book are indeed accurate.

In my opinion, his suit against the blogger is ridiculous. That sort of grotesque language demeans the speaker, but it is still political satire. If the blog were attempting to spread the rumour that Mann is a child molester, that should certainly be libel. Whether the law supposes it libellous merely to compare him to a child molester I don’t know, since the fine points of libel law are so variable, and follow no principles that I understand. But I am sure that I think the law is a ass if it does suppose that. I hope the law won’t sue me for saying that… According to this article, the lawsuit could destroy the august conservative magazine National Review.