Elections, like many things, have some inherent uncertainty. A several point polling error is normal, so a candidate who is down a couple points on election day has a decent shot of winning.
I feel like the correct strategy for 538 when it was actually niche was to be precise, but then it went viral and maybe should've hit the IDK button much harder and more often after that.
On the other hand, it does raise the question how valuable the 538 models for something like this really are if the outcome is a coin flip anyway.
I disagree that it's all pointless though. Most basically it's smart for campaigns to have a good model and let that inform strategy where appropriate. Since the president is a big deal other people's decisions are also impacted, and in the long run it pays to have good predictions of those chances. Also, the outcome sometimes is fairly certain and that isn't always easy to see.
People don't like seeing a 95% chance of winning and then losing. The game tweaks the odds, so certain thresholds become gimmes (something like "if the displayed odds are better than 75%, treat them as 100%").
So I have a bit of sympathy for people who don't have a good intuition for probabilities, given that the world is constantly gaslighting them.
I've even heard things like "70% chance of Hillary winning means she gets 70% of the votes!" (and tangentially, my far-too-long argument with someone on Reddit who insisted "there is no way in hell 50% of the people in this town make above the median income"...)
In the end, it turned out that predicting elections is still very hard, and that for all the fanfare, FiveThirtyEight performed only slightly better than what you could find in any other reputable newspaper, so it kinda lost its appeal.
FiveThirtyEight gave Trump double the odds of the next highest reputable prediction, which was The New York Times Upshot (15%). Princeton Election Consortium gave Trump less than 1%.
That is not "only slightly better" to anyone who's statistically literate.
A FiveThirtyEight reader in 2016 was significantly better calibrated regarding Clinton’s chances than a reader of other reputable newspapers.
People didn't come to 538 for explanations on subtle points of statistical literacy (although those were provided). They came because, for whatever reason, they wanted to know who would win the election.
People not trained in statistics treated like the scoreboard at a football game- it's always better to be winning, but score is a near perfect predictor in the last minute.
Once 538 stopped delivering perfect predictions and started delivering "Actually the difference between 1% and 30% are way bigger than you think" lectures, the appeal disappeared. There are better places to learn math from.
The purpose of FiveThirtyEight was never to be an oracle for the average person. It was always a deliberately wonky site for a wonky audience. They were very clear about that in the articles they published and topics they covered.
----
Nov. 1, 2016 — Election Update: Yes, Donald Trump Has A Path To Victory — https://archive.is/kwdab
> Tuesday was another pretty good day of polling for Donald Trump.
> Trump remains an underdog, but no longer really a longshot: His Electoral College chances are 29 percent in our polls-only model — his highest probability since Oct. 2 — and 30 percent in polls-plus.
> This isn’t a secure map for Clinton at all. In a race where the popular vote is roughly tied nationally, Colorado and New Hampshire are toss-ups, and Clinton’s chances are only 60 to 65 percent in Wisconsin, Michigan and Pennsylvania.
> If you want to debate a campaign’s geographic planning, Hillary Clinton spending time in Arizona is a much worse decision than Trump hanging out in Michigan or Wisconsin.
----
Sept. 16, 2016 — How Trump Could Win The White House While Losing The Popular Vote — https://archive.is/rxP5l
> Using a prototype of a demographic election calculator that FiveThirtyEight will be unveiling in the next few weeks, I decided to simulate a few election scenarios.
> The result? Clinton would carry the popular vote by 1.5 percentage points. However, Trump would win the Electoral College with 280 votes by holding all 24 Romney states and flipping Florida, Ohio, Pennsylvania, Iowa and Maine’s 2nd Congressional District from blue to red.
----
Jun 29, 2016 — Donald Trump Has A 20 Percent Chance Of Becoming President — https://archive.ph/ryIkP
> A 20 percent or 25 percent chance of Trump winning is an awfully long way from 2 percent, or 0.02 percent. It’s a real chance: about the same chance that the visiting team has when it trails by a run in the top of the eighth inning in a Major League Baseball game. If you’ve been following politics or sports over the past couple of years, I hope it’s been imprinted onto your brain that those purported long shots — sometimes much longer shots than Trump — sometimes come through.
----
FiveThirtyEight was probably the worst reputable source to read if you were looking for maximum assurances that Clinton would win.
So maybe we shouldn't be doing it. The value of predicting an election in the large out in public seems kind of dubious, and it's more like gambling than actually being useful. A candidate only runs, and continues running, if they think they can win. All predictions like these do is confuse voters leading up to election day and while they are voting. It keep candidates from making strong cases for their platform, keeps the voters from listening to the candidates' platforms, and encourages team-based partisan politics.
Think that's outweighed by the negatives?
538 was about analyzing and communicating the information from those polls in an easily accessible form. If you came to the site for that, you weren't mad that they "predicted poorly something that was impossible to predict from the data sources they used" ... you were just mad at Trump for winning (despite polls suggesting otherwise).
I thought it went without saying but a good analyst can't predict the future in politics, sports, or anything else. What they can do is make good probabilistic estimates of what is likely to happen. 538 wasn't pretending to do anything more than that.
If people want magic predictions there are plenty of touts and scammers willing to offer them, they don't need to waste time with charts and numbers though.
Well, sure, but how big is the market for that, really? Particularly for a binary outcome like an election, knowing who's going to win is fun, reading a pundit telling you who's going to win can be fun, but ultimately the man in the street is going to take whatever the pundit said and reduce it to candidate X or candidate Y, and you can only do so much better than replacement level at that.
If people are expecting anyone to have a magic prediction algorithm for things like this… I mean there’s only so much one can say. It’s not realistic.
This is especially viewable if you watch them during the 2020 election.
In 2024 the single most likely outcome his model had was trump winning all 7 swing states. The second most likely was Harris winning all 7.
Now you might say that it was on me as a consumer to understand this in 2016, but I remember the look of total shock on Nate Silver’s face when he called the winner on live TV that night, so clearly he didn’t really understand it either. Lesson learned for all of us, I guess.
https://en.wikipedia.org/wiki/FiveThirtyEight#2016_U.S._elec...
A key thing though is 538 did regularly test the calibration their models: https://web.archive.org/web/20190410030104/https://fivethirt...
> "What you’ll find, though, is that our calibration has generally been very, very good. For instance, out of the 5,589 events (between sports and politics combined) that we said had a 70 chance of happening (rounded to the nearest 5 percent), they in fact occurred 71 percent of the time. Or of the 55,853 events that we said had about a 5 percent chance of occurring, they happened 4 percent of the time."
What if they had said 49%? Would that have made their prediction worthless?
Joe Biden on the other hand was a senile wrecker for Build Back Better and the party finally made "the switch" to unelected Harris far too late in the process. Even if she was a great candidate, with her odd laughter and fascination with buses, there was not enough time to shape her candidacy. Her VP candidate choice was hobbled by rising anti-semitism in the party against Shapiro and perhaps concerns of being outshined by him. No, the Democrats did not do themselves any favors in the '24 election.
Carter, Clinton and Obama were media creations, vaulting to national prominence out of nowhere. It helped that Clinton and Obama were great, charismatic choices.
Now the traditional media is fragmented and weak. You're not seeing furtive vaulting attempts for potential phenoms like Newsome gain any traction. Who is the media going to be stuck with next time? Will it be take-two for Harris?
WHEN, not if, Harris loses bigly to Vance, then the Democrats will absolutely be to blame. Where are their all new shiny, beautiful, erudite candidates that would need all four years to gestate and promote? Shouldn't we be getting acquainted with them now? I wager they're not going to appear, and we'll get more flunkies. My theory as to why is that those currently in power in the party do not share; they're aging out and hollowing out the party in the process. We're to the point now of collapse. I'm surprised a third party on the left hasn't yet formed.
Voting in the US, it feels like I am forced to choose between evil and incompetence.
And frankly I’ll vote for incompetence over evil too. Because, y’know, evil.
the turnout-of-demographic-groups-based election model is surely the underlying intelligence failure here.
Surely she must have been in the top 3?
It totally sucks that nobody tried to convince Biden not to run again before the primaries started though.
A model that predicts a 30% chance of winning the election will be wrong 1 out of 3 times, which is not quite a coin flip but close enough.
All he (or anyone) can do is interpret or analyse poll results, and then surface their findings in a way a larger audience can understand. 538 did that better than any other poll analyst ... but they all got it wrong because the polls themselves were faulty.
TLDR; You can't get water from a stone, and no one (not even Nate Silver) can get perfectly accurate predictions from (inherently flawed) polls!
He (or anybody) can make adjustments to the data. He was challenged to explain why his predictions were so different, but he wouldn't do it.
> 538 did that better than any other poll analyst
He made a binary prediction, and it was wrong. There's no such thing as "better" when you only have one outcome. Your prediction is either right or wrong. If by "better" you mean he was wrong but assigned a higher probability to a Trump victory, the best forecaster would have been someone that mechanically changed the probability of a Trump victory to slightly less than 50% no matter what the data said.
But yes I'll join you with the liver damage and drink 17 shots.
538 made very clear with this analogy that both Trump and the Cavs were underdogs, and that both had a solid chance of winning.
If you guessed a "two", and it landed on "two, I wouldn't really be that impressed, even though there was an 83% probability going against you.
FiveThirtyEight had Trump at a 30% chance of winning, and he won. The model wasn't wrong. The less likely of two outcomes occurred. Even if they'd had him at 1% they still wouldn't technically have been wrong though I think complaints might be more warranted.
If they had Trump at 49% would you have still been angry? What about at 51%? Would it have been okay then?
If a coin flip is the necessary mental model to remind you both things can happen, then sure.
People just love horse race coverage. Silver gave us the most accurate horse race coverage. Maybe the lesson is stop following horse race coverage.
But most people went back to the tea leave readers. That way when the election was over, it can justifiably be the charlatan's fault that viewers got over-invested in their predictive capabilities.
But at the same time I do think it’s valid to say it’s more than a coin flip. The polling data over the election cycle showed that Trump had a smaller but still legitimate chance of winning. The data was different in 2020, when he lost.
It doesn't help that the US has a terrible election system that often leads to small margins in some states being decisive.
I know I'm being super conspiratorial here but why wouldn't all forecasters predict just between 30% - 70%? That way if they're "right" they can take the credit for it and if they're wrong they can say "well, we weren't that wrong". That's probably what I'd do anyway...
Also there's more going in those forecasts besides just the "% chance to win". There's expected results in terms of %'s of the vote for the candidates, and that's what people tend to focus on for actually analyzing your performance and credibility after the fact.
You getting the outcome correct but being off by 20 points on the margin is a much worse performance than you getting the outcome wrong but being within 0.5 points of the margin. (ex: Results are 49.75/50.25, you predicted 30/70, another outlet predicted 50.25/49.75).
For anyone making many predictions, you can analyze the outcomes to see how accurate those percentages are.
For anyone making few predictions, you should never trust their track record even if it's technically perfect.
Yep definitely all those.
Why is it so hard to admit 30% is not 0%?
We would need a pass from the mods lol.
What made me mad is Nate seemed to turn into a MAGA troll himself after that election.