Pundits and Polls

(This is my second post of tht weekend and I hope new visitors will check the one below as well.)

I was in Washington this weekend and I saw some interesting columns in and stories in the Washington Post. As my regular readers know, I rely mainly--indeed, almost totally--for electoral intelligence on Nate Silver at the New York Times's fivethirtyeight.com. Silver started his career writing on baseball at baseball-prospectus.com, and his reception among political pundits, most of whom must make at least ten times the money that he does, is similar to the great sabermetrician Bill James's reception among sportswriters, none of whom made nearly as much money as James did from his Baseball Abstracts in the 1980s. Both pundits and sportswriters cultivate their readers by claiming unique insights and inside knowledge. They instinctively seem to resent the idea that a patient, scientific observer could, simply by crunching numbers, reach sounder conclusions than they could. This remains the attitude of the Boston press towards James, even though he has clearly been a key figure in the Red Sox Renaissance over the last 10 years--a period which, sadly, seems to have come to an end despite his best efforts. They were livid a few years ago, for instance, when James once again demonstrated pretty clearly that Jim Rice didn't belong in the Hall of Fame. He was right, but the sportswriters elected him anyway on the grounds that he was "the most feared hitter" of his generation, the kind of quality they love precisely because it doesn't translate into superior statistics.

Silver has a data base consisting essentially of every state and national poll taken for at least the last 35 years, and perhaps more. This allows him to find out how many times the average of all available polls has been correct. It also allows him to adjust for systematic bias in particular polling outfits, and to estimate very accurately the chances that today's polls will in fact predict the winner, even if the election is very close. Karl Rove, who is a political professional but too obviously partisan, it seems to me, to be counted as a pundit, employed the opposite technique in his Wall Street Journal op-ed a few days ago to predict a Romney victory. He simply grabbed ever hopeful piece of data he could find, vastly exaggerated its significance, and concluded that Romney was going to win. This technique is common among political junkies. My late father, for instance, having noticed in one election that a Democratic candidate had overcome a three-point polling deficit to win his race, assumed for the rest of his life that any Democrat trailing by three points or less was going to win. Rush Limbaugh did this kind of statistical cherry-picking on Friday as well: "The only thing that you can say about these polls is that in none of them is Obama at 50. And that matters. When the incumbent is not at 50, I mean, that's a rule of thumb. So is the rule of thumb about he who wins independents wins the election. I don't care what the overall result is, in all the polls it's Romney up, for the most part, double digits. Not in all, but in many of them double digits in independents." Nice try, Rush.

This morning Dana Millbank, in my opinion, embarrassed himself pretty badly by reviewing Silver's projection, which currently gives Obama an 85.1% chance of winning the election and expects him to win with about 307 electoral votes. Millbank started his column well by showing that we could discount Rove based upon his track record--he has consistently exaggerated the Repubicans' chances in every election since 2000 except 2010, it seems--but he then turned Silver and declared that he, Millbank, gave Silver "a 50-50 likelihood of being correct. The truth is anybody who claims to know what is going to happen on Election Day is making it up and counting on being lucky." That statement is ludicrous. Silver is the first to admit that Romney could win the election, but the chances of him doing so are, according to the best model we have--Silver's--15%, not 50%. (In fact, Silver's probability of Obama's victory has gone up 10 points since Millbank wrote his column). I've never met Millbank and I don't know why he can't see this. It's not as if Silver doesn't have a track record: he called the last Congressional elections almost perfectly, a much more difficult feat, it seems to me, with 435 separate elections to deal with instead of 51. (I'm very disappointed that the Times editors allowed him to ignore the House completely this year.) But Millbank didn't embarrass himself nearly as badly as George Will, for whom the results are likely to be a huge disappointment. Will lists a series of possible events Tuesday and assigns them certain significance. If Romney wins Wisconsin, he says, that will prove how popular Governor Walker's union-busting strategy was. It might, indeed, prove that, but Silver's analysis of the polls shows that Obama has a 94.6% chance of carrying Wisconsin. Will adds that if Romney wins Pennsylvania, "or even comes close," it will show Republicans to be more popular than expected among the elderly, since Pennsylvania has one of the oldest populations in the nation. In fact Will must know that Republicans are already more popular than Democrats among the elderly and will almost surely carry the over-50 vote and the over-70 vote by an even greater margin--and he should know that according to Silver, Obama's chances in Pennsylvania are 97.3%.

There is another far more important political lesson to learn from James and Silver: the truth is out there, on a lot of subjects, if you're willing to gather data and analyze it dispassionately. We could do for health care and taxes and yes, maybe even for global warming, what they have done for baseball and elections if we wanted to--but those are areas where there's too much money at stake for reasoned analysis to prevail, certainly at times like these. Rove, Will and Millbank are all part of the power elite. Nowadays, it isn't very interested in the facts.