Who speaks for statistics?
Ace forensic psephologist Nate Silver has attracted quite a bit of attention lately, with his 4+-year-old blog devoted to his statistical model that is intended to provide a synoptic view of the entire range of public data to produce a single probabilistic prediction of the outcome. Now, there are some clear criticisms that could be made of his approach, and of his results — in particular, the obvious failure of his successive predictions to be martingales, as they would have to be if they were appropriately using all current information — but he has been remarkably clear and open about his procedures and principles, and his reasoning on matters large and small seems generally sound, if not necessarily compelling. It’s funny that his conclusions should arouse any controversy at all, given that they are hardly different (as Silver himself is quick to acknowledge) from the conclusions one would draw from a simplistic combination of poll results. His main contribution is in giving careful answers to the obvious critiques that could be proposed: What’s a reasonable estimate for the difference between state poll results and the actual election result? How correlated are polling errors? What’s the best way to average polls of varying qualities done over multiple days? And so on. In the end, the answer doesn’t differ much from what anyone with number sense would come up with in a few hours, but you don’t know that for sure until you do it. And Silver’s reputation derives from the sense and good care that he takes in posing these questions and resolving them.
(The failure of the martingale property is actually evidence of his honesty in following the model that he set up back in the spring. He clearly would have been capable of recognising the trends that other people can see in the predictions, and introducing an ad hoc correction. He didn’t do that.) Continue reading “The Silver standard”