Improbable things happen

Cogito ergo sum Logic and rhetoric

Key articles

Logical fallacy Syllogism Argument

General logic

Appeal to emotion Anecdotal evidence Regression to the mean Apophasis Burden of proof Pensamento mágico

Bad logic

Danth's Law Demonization Spotlight fallacy Ecological fallacy Gambler's fallacy Affirming the consequent v - t - e

“”That a particular specified event or coincidence will occur is very unlikely. That some astonishing unspecified events will occur is certain. That is why remarkable coincidences are noted in hindsight, not predicted with foresight.

—David G. Myers

Improbable things happen all the time.

Creationists (e.g., William Lane Craig) and all manner of non-rationalists like to disparage their opponents or bolster their own arguments by pointing out the lack of probability of something happening. Out of all the possibilities, they say, this one is the one that occurred — how fantastically unlikely and amazingly miraculous! It is simply impossible to believe that it just happened by chance!

But improbable things happen all the time, because there are many improbable events that could happen, so it is not surprising if some improbable event does happen, even though for any event that is fixed à priori, it is very unlikely to happen. The trouble is that we must differentiate between (a) "This particular improbable pattern of lottery numbers came up on this particular day in this particular lottery" and (b) "Some improbable pattern of lottery numbers came up sometime in the last five years somewhere in the world."

In short: "improbability" does not imply "impossibility". On the other hand, when a string of improbabilities connect with each other in a large sequence of correct choices among an almost infinite series of wrong choices, which in turn lead to a useful outcome, then it is rational to conclude there was no randomness. In short: it would not be rational to believe it a random event when winning the lottery 10 times in a row.

Fat chance

The lottery

Possibly the simplest example is a lottery. These often have incredible odds that seem impossible to beat, but indeed someone (almost) always wins. This is because of the sheer number of people playing. Even though an individual has a low chance of success, overall it's almost certain that it will be won by somebody. Most people will refrain from submitting a ticket with six sequential numbers because of the rationalisation that such a draw is too improbable — despite the fact that all draws are equally likely.

The idea of this can also be expressed by looking at car licence plates. Imagine seeing one with the configuration HJB-546. That's one out of a combination of over 17 million, so it seems like a remarkably improbable feat if you treat it in the same way as the statistically illiterate. But any combination is equally improbable, and you're certain to see one of the combinations if you look for it. It would only become remarkable if you predicted the configuration in advance.

As Richard Feynman once quipped:

“”You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won't believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance I would see that particular one tonight? Amazing!

The same birthday

Consider a party attended by thirty people: what are the chances that two of them have the same birthday (ignoring leap years, and assuming the attendees' birthdays are entirely random)? One in twelve, or, roughly 8% (30/365)? After all, it's a 1 in 365 chance someone will share your birthday, and by the lottery analogy above, there's 30 shots at winning.

No, the odds are significantly better than that. In fact, there is a 70% probability.

This is known as the "birthday problem". The apparently miraculous breaking of odds is attributed to the fact that the question is "what is the chance that any two people have the same birthday?", whereas most people following common sense tend to translate the question as "what is the chance that someone has the same birthday as mine?". So while you get 30 shots at this 1 in 365 lottery, so does everyone else. More specifically, every possible pairings of two individuals in the group of 30 has a shot at this 1 in 365 chance. Regardless, the answer is very non-intuitive and is a good display of how people don't do well at guessing probabilities. Once the problem is known, however, calculating the real odds is just a simple case of exploiting the correct mathematics.

Shuffling a deck of cards

Do you want to witness an "improbable" event right now in your very own home?

Take a standard deck of 52 cards, shuffle it well and spread the cards in a line. Look at them well. Assuming an ideally random shuffle, the probability of a card sequence in this exact order is…

1 in 80658175170943878571660636856403766975289505440883277824000000000000.

Really. And yet despite this very low probability, you just got that sequence. Which may be mindblowing if you haven't studied statistics or combinatorics. Of course, this is because the probability that is given to you is ex ante and when you are reading the sequence of the cards after you shuffle them, you are simply validating what you see. The ex post probability of getting that particular sequence is always 100%.

Ourselves

It is wildly improbable that any person alive is — well, alive. To be alive, for instance yours truly sitting in front of a computer and writing, is so improbable a priori, that "Borel's Law" (poor Borel) rules out my existence. For countless generations a particular sperm cell had to encounter a particular egg and produce every single descendant — and that goes for all the other lines as well. The a priori probability is staggeringly low — and yet, here I am. According to "creationist probability" I shouldn't exist.

Beatrice, Nebraska

Beatrice, Nebraska, is probably not widely known, but a miracle took place there in the evening of March 1, 1950. The church choir was due to meet at 7:20 p.m. All 15 members were late due to 10 separate reasons. The church exploded at 7:25 p.m. The members obviously wondered about God's hand in this.

Mathematics-filled analysis for people who inexplicably like math

See also Littlewood's law, Ramsey theory

Pulling all the aces in four attempts from a full deck is 1 in 270,725. Pulling at least one ace from a full deck in four attempts is actually about 3 in 10.

One example in statistics is the phenomenon of "at least one". Imagine 6 cards are laid out face-down and the only certainty is that 2 cards are aces and 4 cards are not aces. What many people will assume from intuition is the chance of choosing at least one ace when flipping two cards over is 2 in 6 (~33%). This is only true of drawing an ace on the first attempt, however. The actual chance of picking up at least one ace is much better than that.

This works because the probability of at least one is equal to 1 minus the probability of none, and it's this calculation that needs to be made. It might seem backwards — because it is — but this is the easiest way to calculate an "at least one" probability as this also includes the chances of drawing more than one automatically. In this case the probability of not drawing any aces can be determined with the formula P(A)*P(B|A), which is read as "the probability of A multiplied by the probability of B assuming that event A already occurred". P(A) is the probability of not turning over an ace out of 6 cards, and P(B|A) is the probability of not turning over an ace out of five cards assuming you didn't the first time (since there is no replacement of the first card). This quite clearly gives us the odds of not turning over an ace across two attempts and is all we need to solve the problem. So, P(A)*P(B|A) would work out as (4/6)*(3/5), which equals 12/30, or 40%. Therefore, we can conclude that the odds of drawing "at least one" ace is actually 60%.

Working from a full deck of 52 illustrates why this backwards method for "at least one" works more efficiently. To do the calculation forward you would have to calculate and combine the individual odds of drawing one, two, three and four aces in different combinations. For example, drawing an ace on the second attempt is different as you're drawing from 51 cards, not 52 so you need to calculate (48/52)*(47/51) and add it to a stack of other possible combinations. This becomes increasingly complicated and only becomes more so if you start increasing the number of attempts. On the other hand, it's only a single calculation to work out the probability of drawing no aces. This is (48/52)*(47/51)*(46/50)*(45/49), about 0.72. So the probability of drawing at least one ace in four attempts is 0.28, about 3 in 10 — remarkably good odds for such a "rare" card.

This is similar to the case described above of many players playing the lottery. The 2 in 6 odds are true for any one selection. But if we were given a second chance to play again from scratch and at least one had to successfully draw an ace, these odds would additively combine to 4 in 6, or ~67%.

Actual uses

The System

The effect was exploited in Derren Brown's TV special "The System," where he presented a system for winning bets placed on multiple race horses. He began with several thousand volunteers and then subsequently only followed the winners; the final product that was televised only featured one individual, making his "system" seem miraculous. To demonstrate the system, he also performed the coin tossing trick, taking around 9 hours to film all of his attempts until he did come up with a successful combination.

Paul the Octopus

He just got lucky, that's all.

A similar thing happened in the 2010 South Africa World Cup, when Paul the Octopus was thought to have predicted the outcome of eight matches. A great part of the real explanation is very simple: there was a 1 in 256 probability that Paul could predict the outcome of eight games, and Paul just casually happened to be that one in 256 that was reported in the media. (Magical thinking, of course, processed this fact as Paul being a psychic octopus.)

Large sporting events like the World Cup generate masses of interest and undoubtedly many people will try to predict the outcome — in fact, it would be unlikely that an event this size should attract less than the 256 people or processes required to statistically guess the 8 matches correctly. Similar to the Derren Brown example discussed above, this shall be self-selecting. Only a fraction shall guess the first game correctly, a fraction of those shall guess the second and so on. By the time it's whittled down to the last few games (not unlike a football tournament, of course) people might be gathering attention as being "on a lucky streak". Naturally, the ones who fall at the final hurdle lose their streak, while the winners emerge as skilled, or psychic.

The main difference between sports betting and the other examples above, however, is that the odds are not mathematically perfect. Teams have different levels of performance and ranking, and favourites are very likely to emerge. As a result, it's never really a 50:50 chance for any team entering a match — seriously, ask any bookies to give you evens on Brazil v England and they will laugh in your face. As a result, for most people keen on the sport it's actually a little under the 1-in-256 odds required to guess 8 games in a row. This only goes to convert an apparently improbable performance of prediction into a dead certainty.

Redskins Rule

The Washington Commanders, known at the time as the Redskins, moved to Washington, DC in 1937. For the next 16 US presidential elections (1940 through 2000), the following rule remained true:

If the Redskins win their last home game before the election, the party that won the previous election (the incumbent party) wins the next election. If the Redskins lose this last home game, the incumbent also loses and the challenging party's candidate wins.

Folklore had established this rule by the early 1990s, but only became widely known around 2000. (Since 2004, the Redskins Rule seems to have inverted, with the incumbent party winning whenever Washington lost in four of the five elections up to 2020, only holding true in 2008 when challenger Barack Obama won the presidency despite Washington losing).

Since it was brought to light in 2000, only one of five elections obeyed the rule at all — demonstrating past observations don't affect future probabilities. This is a firm example of post hoc reasoning via selection. There are dozens of teams in the NFL (add to that the NBA, NHL, and so on…) and so the odds of at least one of these teams' results syncing up with the election is more modest than you'd think. Certainly, if the rule didn't hold true, it shouldn't be reported. Much like in the System, above, the rule is self-selecting, as fewer teams — since the 1930s — would sync up with the election so well. For instance, if we start in the 1932 election between Herbert Hoover and Franklin D. Roosevelt, then about half of all teams playing in the 1932 season would have won their last home game, and obeyed the rule quite well. From there, it's a trivial case of letting random chance converge on a team that correlates quite well.

Flawlessly predicting everything

Andy Baio came up with a method that would allow anyone to destroy trust in American democracy, by claiming to be a voting machine manufacturer and predicting the state-by-state results of a US presidential election on Twitter, six months in advance. The way to do this is to create a private Twitter account, post tweets with every possible outcome six months in advance, then wait till after the election when you delete all the wrong predictions. You now make public your Twitter account and show everyone your predictions dated 6 months previously, because nobody can change a tweet's date. You demonstrate that a voting machine manufacturer knew the election result in advance, proving the election was fixed, and creating chaos. This hasn't actually been done in real life (possibly Twitter would intervene eventually). But it has been used more innocently by someone pretending to be a fake insider with all the latest video games news — potentially good for getting Twitter followers, but unlikely to provoke insurrection.

The Law of Truly Large Numbers

When talking about the improbable, it's easy to ignore the cases where the event does not happen. People are naturally self-centered and think about their own experience first: from any one individual's point of view, the odds of winning the lottery are minuscule and the odds of finding someone with the same birthday are exactly as you'd expect.

But when considered in a more comprehensive and inclusive way, the true odds are revealed. For example, the probability of one particular mutation during evolution may be tiny, but there are billions of mutations happening continuously and being sorted by natural selection. Because of all these chances, that one minute possibility isn't really unlikely at all. It's a certainty.

We tend to pay attention to the improbable things that do happen and never to the improbable things that don't happen and don't defy the odds. This particular cognitive bias is an important aspect of the Black Swan theory of improbable events. We may be staggered by an event with a 1-in-a-million odds, but completely ignore that at least 999,999 other 1-in-a-million events just happened to have not occurred. This is often boosted by a form of post hoc fallacy that explains the event that happened but discounts the events that don't, analogous to rolling a die but only ever telling someone or acknowledging the roll when it's a 6; indeed, the die may be invisible and no one knows it's being rolled until it shows a 6.

In short, because there are millions of events that one can observe or take note of, one should not be surprised to see observe or take note of *some* one-in-a-million event, even though any particular given one-in-a-million event is very strongly not expected to occur.

See also

• Anecdotal evidence
• Appeal to probability
• Borel's Law, a rough statistical rule of thumb often abused by creationists in argument.
• Common sense
• Confirmation bias
• Gambler's fallacy
• Fun:The Hitchhiker's Guide to the Galaxy
• Littlewood's law
• Magical thinking
• Statistical significance
• Texas sharpshooter fallacy, the related fallacy where the same data are used to both construct and test a hypothesis.

External links

• Another amazing feat of improbability that makes winning the lotto seem likely in comparison
• 1-in-a-Trillion Coincidence, You Say? Not Really, Experts Find