| How the sausage is made|
Nathaniel Read "Nate" Silver is probably a witch. After making a fuck-ton of money through online poker (in its relative infancy when it mostly attracted people with more money than poker skill) he made a name for himself by becoming a political statistician, founding the blog FiveThirtyEight to analyse US elections and becoming a staple on the New York Times. FiveThirtyEight has since been moved to a larger partnership with sports network ESPN.
In contrast to pundits, Silver's approach was to use both mathematics and evidence to make his predictions — notably the use of Bayesian statistics, which take into account prior information on how states voted. Putting it down to being a bit of a nerd for numbers, this has turned out to be remarkably successful.
With a combination of this skill[Note 1] and a bit of luck, he successfully predicted the outcome of the 2012 U.S. Presidential Election with almost perfect accuracy. This was notable because at the time most mainstream media were predicting the election to be "too close to call" — Silver, on the other hand, actually looked at the data and found it to tell a different story, one where the incumbent Barack Obama had a 90% chance of winning, and by a significant margin. It was the second time he pulled this off, getting 49/50 states right in 2008.[Note 2]
Silver, while mostly accurate in his predictions, made several rather uncharacteristic mistakes during the 2016 elections. While he indicated that Hillary Clinton would win the Michigan Democratic primary, it actually went to Bernie Sanders - although virtually every pollster got that one wrong, so it's not entirely his fault.
What is his fault, however, is when he endorsed and co-wrote an article titled "Bernie Sanders may win Iowa and New Hampshire - and lose everything else," which certainly wasn't very accurate at all. Another fuck-up occurred when he insisted that Donald Trump would not win the Republican presidential nomination despite repeated polling evidence showing that Trump was leading; Silver himself admitted that he was dead wrong about this one. He's since admitted using punditry for this election cycle rather than just sticking to his raw-numbers analysis.
538's model uses high correlations between the states, to the extent that a candidate winning a single swing state doubles their probability of winning the election. For example, if Clinton wins Nevada, in the model her probability of winning goes up to 91, not because the 6 electoral votes from Nevada are so very important, but because according to Nate's model, the swing states behave in a very similar manner, so whoever wins one swing state will sweep all of them. In this case, that obviously favored Trump. He needed to win all swing states plus at least one "blue wall" state. FiveThirtyEight's final prediction for the 2016 election had Clinton's probability of winning at about 70% and Trump's at 30%, but Trump ended up winning because he did in fact pull off the once-thought-to-be-impossible feat of winning most of the swing states (including, crucially, Florida and Pennsylvania) as well as two other states thought to be solidly Democratic (Wisconsin and Michigan, albeit winning the latter very narrowly indeed). Clinton did win Nevada, but losing those key northern states destroyed her once infallible position in the Northeast, and that coupled with the loss of Florida lost her the election. Therefore, Silver's predicted result for the election using this model was way off, although he was correct that one candidate was going to sweep most of the swing states - it's just that he picked the wrong one. It should be mentioned however, that Silver frequently pointed out that Trump was just "one standard size polling error" away from winning the electoral vote. And the national popular vote (which went to Clinton by two points) was well within the margin of error (the polls had her up by roughly four points). In fact in a "Dewey defeats Truman" election, Silver was pretty much the only mainstream source without partisan bias or motivation to give Trump more than a "Yeah, and pigs can fly" chance on election day. But because Democrats were so shocked at losing the election after "everybody" had them comfortably winning it and Republicans had had an axe to grind with him for 2012 and combined with the general public apparently being incapable of grasping the likelihood of a "30%" event actually happening - roughly one in three times - Silver and his website now has to deal with hecklers saying "Yeah and you were wrong on the 2016 election, so shut up".
This was the worst possible result for Nate Silver personally. He correctly warned of a split result between the Electoral College and the popular vote, and repeatedly said that Trump had a better chance at winning than most gave him credit for (compare The Huffington Post claiming that Clinton had a 98% chance of winning). However, because his model failed to predict the actual winner, Silver's reputation as an election-forecasting wizard has arguably gone down the tubes. Instead of being remembered for correctly predicting all 50 states between Obama and Romney, he may be remembered for his methodology (like everyone else's) failing to predict the final 2016 presidential result. Also, the sheer avalanche of election "updates" (usually overly-long blog posts stating the obvious) gave the impression that fivethirtyeight.com was massaging the "horse race" for clicks.
As Silver was both predicting a win for Obama and turned out to be staggeringly correct about it, he has become something of a pariah and figure of hate for those on the US far right, who still hold their heads in the sand over the election. Being openly gay certainly hasn't helped there, as comments from Dean Chambers, who runs unskewedpolls.com, seemed almost entirely ad hominem in nature — dismissing Silver as nothing more than an "effeminate," "soft-sounding" "Castrat[o]."
Predicting the world
He is described by Black Swan author Nassim Nicholas Taleb as "seriously knowing his shit,"[Note 3] although the two appear to disagree on how predictable the world actually is. In his book The Signal and the Noise, Silver claims that the 9/11 terrorist attack was, to a degree, predictable — and furthermore that statistics may suggest another, even larger, attack is coming within the next decade or so — while Taleb's Black Swan theory would hold that such events are outside the realm of predictability.
- Arthur C. Clarke's Third Law
- Not that it's difficult, just very uncommon amongst the usual political talking heads.
- In both cases, "he successfully predicted the outcome" means that the candidate Nate Silver gave the highest probability of winning won the given state. Silver's predictions (and others') are probabilistic, so over enough elections the prediction should often be "wrong" - for example, if it's predicted that a candidate has a 60% chance of winning in each of ten states, over enough elections the candidate should lose an average of about four.
Unfortunately the US presidential election does not happen often enough to really assess the accuracy of prediction methods, and the popular media (and the public) do not understand statistics well enough to distinguish "Candidate A has a 60% chance of winning Virginia" from "Candidate A is guaranteed to win Virginia".
- Okay, so the actual quote is "He's the real deal."
- As of press time, Nate Silver is probably a witch.
- And the Times has filled in the vacuum with The Upshot.
- Nate Silver: it's the numbers, stupid, The Guardian
- Nearly 50 high-profile wingnuts.
- [Gawker.com: archive.is, web.archive.org Shooting the Messenger’s Numbers: Nate Silver’s Struggle (The Redux)], Gawker