Improbable things happen
(→The System) |
m (Bare wikipedia.org links -> WP: links by (partially working) Wikilinkify script) |
||
| Line 12: | Line 12: | ||
Possibly the simplest example is a lottery. These often have incredible odds that seem impossible to beat, but indeed ''someone'' (almost) always wins. This is because of the sheer number of people playing. Even though ''an individual'' has a low chance of success, overall it's almost certain that it ''will'' be won by ''somebody''. Most people will refrain from submitting a ticket with six sequential numbers because of the "rationalisation" that such a draw is too improbable - despite the fact that all draws are equally likely. | Possibly the simplest example is a lottery. These often have incredible odds that seem impossible to beat, but indeed ''someone'' (almost) always wins. This is because of the sheer number of people playing. Even though ''an individual'' has a low chance of success, overall it's almost certain that it ''will'' be won by ''somebody''. Most people will refrain from submitting a ticket with six sequential numbers because of the "rationalisation" that such a draw is too improbable - despite the fact that all draws are equally likely. | ||
| − | The idea of this can also be expressed by looking at car licence plates. Imagine seeing one with the configuration [ | + | The idea of this can also be expressed by looking at car licence plates. Imagine seeing one with the configuration [[WP:File:SASKATCHEWAN_1977_plate_(2156625545).jpg | HJB-546]]. That's one out of a combination of over 17 million, so it seems like a remarkably improbable feat if you treat it in the same way as the statistically illiterate. But any combination is equally improbable, and you're certain to see one of the combinations if you look for it. It would only become remarkable if you predicted the configuration in advance. |
==Same birthday== | ==Same birthday== | ||
Revision as of 16:31, 9 February 2012
| Part of the series on |
| Key articles |
| General logic |
| Bad logic |
Improbable things happen, all the time.
Creationists, anti-evolutionists and all manner of non-rationalists like to disparage their opponents or bolster their own arguments by pointing out the improbability of something happening. But, improbable things happen all the time, because the "improbability" is an illusion based on our preconceptions, and not a statistical truth.
In short, "improbability" does not imply "impossibility".
According to the ergodic hypothesis, provided the ultimate laws of physics possess certain reasonably possible (but unproven) characteristics, then every event with non-zero probability, however small, will eventually occur.
Contents |
The lottery
Possibly the simplest example is a lottery. These often have incredible odds that seem impossible to beat, but indeed someone (almost) always wins. This is because of the sheer number of people playing. Even though an individual has a low chance of success, overall it's almost certain that it will be won by somebody. Most people will refrain from submitting a ticket with six sequential numbers because of the "rationalisation" that such a draw is too improbable - despite the fact that all draws are equally likely.
The idea of this can also be expressed by looking at car licence plates. Imagine seeing one with the configuration HJB-546. That's one out of a combination of over 17 million, so it seems like a remarkably improbable feat if you treat it in the same way as the statistically illiterate. But any combination is equally improbable, and you're certain to see one of the combinations if you look for it. It would only become remarkable if you predicted the configuration in advance.
Same birthday
Consider a party attended by thirty people: what are the chances that two of them have the same birthday? One in twelve, or, roughly 8% (30/365)? After all, it's a 1 in 365 chance someone will share your birthday, and by the lottery analogy above, there's 30 shots at winning.
No, the odds are significantly better than that. In fact, there is a 70.6% probability. This is known as the "birthday problem". The apparently miraculous breaking of odds is attributed to the fact that the question is "what is the chance that any two people have the same birthday?", whereas most people following common sense tend to translate the question as "what is the chance that someone will have the same birthday as me?". So while you get 30 shots at this 1 in 365 lottery, so does everyone else. More specifically, every possible pairings of two individuals in the group of 30 has a shot at this 1 in 365 chance. Regardless, the answer is very non-intuitive and is a good display of how people don't do well at guessing probabilities. Once the problem is known, however, calculating the real odds is just a simple case of exploiting the correct mathematics.[1]
At least one
Another example in statistics is "at least one". Imagine 6 cards are laid out face-down and the only certainty is that 2 cards are aces and 4 cards are not aces. What many people will assume from intuition is the chance of choosing at least one ace when flipping two cards over is 2 in 6 (~33%). This is only true of drawing an ace on the first attempt, however. The actual chance of picking up at least one ace is much better than that.
The probability of at least one is equal to 1 minus the probability of none. It might seem backwards - because it is - but this is the easiest way to calculate an "at least one" probability as this also includes the chances of drawing more than one automatically. In this case the probability of not drawing any aces can be determined with the formula P(A)*P(A|B), which is read as "the probability of A multiplied by the probability of B assuming that event A already occurred". P(A) is the probability of not turning over an ace out of 6 cards, and P(A|B) is the probability of not turning over an ace out of five cards assuming you didn't the first time (since there is no replacement of the first card). This quite clearly gives us the odds of not turning over an ace across two attempts and is all we need to solve the problem. So, P(A)*P(A|B) would would work out as (4/6)*(3/5), which equals 12/30, or 40%. Therefore we can conclude that the "at least one" ace odds is actually 60%.
Working from a full deck of 52 illustrates why this backwards method for "at least one" works more efficiently. To do the calculation forward you would have to calculate and combine the individual odds of drawing one, two, three and four aces in different combinations. For example, drawing an ace on the second attempt is different as you're drawing from 51 cards, not 52 so you need to calculate (48/52)*(4/51) and add it to a stack of other possible combinations. This becomes increasingly complicated and only becomes more so if you start increasing the number of attempts. On the other hand, it's only a single calculation to work out the probability of drawing no aces. This is (48/52)*(47/51)*(46/50)*(45/49), about 0.72. So the probability of drawing at least one ace in four attempts is 0.28, about 3 in 10 - remarkably good odds for such a rare card.
This is similar to the case described above of many players playing the lottery. The 2 in 6 odds are true for any one selection. But if we were given a second chance to play again from scratch and at least one had to successfully draw an ace, these odds would additively combine to 4 in 6, or ~67%.
The amazing coin predictor
Imagine someone who could predict the random toss of a coin ten times in a row. Surely that person must be psychic or incredibly lucky?
No, they are just happen to be the random 1 in 1000 for whom the statistics are favorable.
If we have 1000 people, or for the sake of exact numbers 1024 people, who are asked to predict the toss of a coin we can illustrate how it works. Assuming half will always call tails and the other half will call heads then we would expect 512 will get it right once - of course, these are idealised odds as this is a thought experiment, in real life we'd expect some natural variance. On a second toss, half (256) of the first group will guess right again. Subsequent tosses will halve the number of people who get it right until after 10 tosses one person will have been right 10 times in a row. On such statistical streaks are the careers of stock-pickers and other charlatans built.
The System
This effect was exploited in Derren Brown's TV special The System, where he presented a system for winning bets placed on multiple race horses. He began with several thousand volunteers and then subsequently only followed the winners; the final product that was televised only featured one individual, making his "system" seem miraculous. To demonstrate the system, he also performed the coin tossing trick, taking around 9 hours to film all of his attempts until he did come up with a successful combination.
Paul the Octopus
A similar thing happened in the 2010 South Africa World Cup, when Paul the Octopus was thought to have predicted the outcome of eight matches. A great part of the real explanation is very simple: there was a 1 in 256 probability that Paul could predict the outcome of eight games, and Paul just casually happened to be that one in 256 that was reported in the media. (Magical thinking, of course, processed this fact as Paul being a psychic octopus).
Large sporting events like the World Cup generate masses of interest and undoubtedly many people will try to predict the outcome - in fact, it would be unlikely that an event this size would attract less than the 256 people or processes required to statistically guess the 8 matches correctly. Similar to the Derren Brown example discussed above, this will be self selecting. Only a fraction will guess the first game correctly, a fraction of those will guess the second and so on. By the time it's whittled down to the last few games (not unlike a football tournament, of course) people might be gathering attention as being "on a lucky streak". Naturally, the ones who fall at the final hurdle lose their streak, while the winners emerge as skilled, or psychic.
The main difference between sports betting and the other examples above, however, is that the odds are not mathematically perfect. Teams have different levels of performance and ranking, and favourites are very likely to emerge. As a result, it's never really a 50:50 chance for any team entering a match - seriously, ask any bookies to give you evens on Brazil v England and they will laugh in your face. As a result, for most people keen on the sport it's actually a little under the 1 in 256 odds required to guess 8 games in a row. This only goes to convert an apparently improbable performance of prediction into a dead certainty.
Shuffling a deck of cards
Do you want to witness an "improbable" event? Take a standard deck of 52 cards, shuffle it well and spread the cards in a line. Look at them well. Assuming an ideally random shuffle, the probability of a card sequence in this exact order is...
1 in 80658175170943878571660636856403766975289505440883277824000000000000
Really. And yet despite this very low probability, you just got that sequence. Which may be mindblowing if you haven't studied statistics or combinatorics. Of course, this is because the mathematical odds of ending up with any 52 card sequence is 100%.
As Richard Feynman once quipped:
“”You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won't believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance I would see that particular one tonight? Amazing!
|
General observations
The general rule when looking at how apparently improbable things happen is that if you make multiple attempts, you can't ignore the cases where the event does not happen. It's easy to neglect this because people are naturally self-centred and will think about their own experience first. From any one individual's point of view the odds of winning the lottery are minuscule and the odds of finding someone with the same birthday are exactly as you'd expect. When considered in a more comprehensive and inclusive way, the true odds are revealed. The odds of a particular beneficial mutation to drive evolution may be minute, but there are billions of mutations happening continuously and are non-randomly sorted via natural selection. Because of this, that one minute chance isn't really a minute chance but a near certainty.
We're also massively biased towards paying attention to the improbable things that do happen - and never to the improbable things that don't happen and don't defy the odds. This particular cognitive bias is an important aspect of the Black Swan theory of improbable events. We may be staggered by an event with a 1 in a million odds, but completely ignore that at least 999,999 other 1-in-a-million events just happened to have not occurred. This is often boosted by a form of post hoc fallacy that explains the event that happened but discounts the events that don't - analogous to rolling a dice but only ever telling someone or acknowledging the roll when it's a 6, indeed the dice may be invisible and no one will know it's being rolled until it shows a 6.
In short, 1-in-a-million chances happen 9 times out of 10.
See also
- Borel's Law - a rough statistical rule of thumb often abused by creationists in argument
- Magical thinking
- Confirmation bias
- Gambler's fallacy
- Conservapedia:Probability
- Littlewood's law
- Statistical significance