|
Nominations and Campaigning for the 2024 RationalWiki Moderator Election are now open |
Extraordinary claims require extraordinary evidence
| This might be Skepticism |
|
| But we're not sure |
| Who's asking? |
"Extraordinary claims require extraordinary evidence" (a.k.a., the Sagan standard) was a phrase made popular by Carl Sagan. Its roots are much older, however, with the French mathematician Pierre-Simon Laplace
stating that: "...the more extraordinary a fact, the more it needs to be supported by strong evidence"[note 1] Also, David Hume wrote in 1748: "A wise man … proportions his belief to the evidence", and "No testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous than the fact which it endeavors to establish."[2] and Marcello Truzzi says: "An extraordinary claim requires extraordinary proof."[3]
Either way, the phrase is central to the scientific method, and a key issue for critical thinking, rational thought, and skepticism everywhere.
The evidence put forth by proponents of such things as gods, ghosts, the paranormal, and UFOs is highly questionable at best and offers little in the way of proof. Even if we accepted what evidence there is as valid (and it is highly debatable if we should), limited and weak evidence is not enough to overcome the extraordinary nature of these claims.
Analogy[edit]
Alice and Bob are two friends talking after school. Alice tells Bob that she watched a movie the previous evening. Bob believes her easily because he knows that movies exist, that Alice exists, and that Alice is capable and fond of watching movies. If he doubts her, he might ask for a ticket stub or a confirmation from one of her friends. If, however, Alice tells Bob that she flew on a unicorn to a fairy kingdom where she participated in an ambrosia-eating contest, and she produces a professionally-printed contest certificate and a friend who would testify to the events described, Bob would still not be inclined to believe her without strong evidence for the existence of flying unicorns, fairies, and ambrosia-eating contests.
Probability theory[edit]
While the idea that a sufficiently outlandish claim requires a lot more compelling evidence is quite intuitive, it can be quantified nicely with probability theory in a Bayesian framework. In short, sufficient evidence must be capable of raising a highly improbable claim to be highly probable — and the more improbable the evidence, the better. By application of Bayes' theorem, it's possible to show this in action mathematically.
Assume, for instance, someone claims to be able to predict what way a coin[note 2] will land almost perfectly. We know this is an extraordinary claim, so we'll say that just by guessing if the person is telling the truth or not that it's a million-to-one chance. In reality, the number would be even more improbable, but this can be used for illustration. So we ask them to demonstrate the skill. They're almost perfect, so let's assume they guess right about 90% of the time — this allows them the opportunity for their skill to mess up once in a while, but still prove to be pretty good. This gives us all the information we need to know to quantify how extraordinary the evidence must be.
Consider if they guessed a single coin toss correctly. The odds of guessing by chance is a mere 50%, or 50:50.
A single coin toss doesn't improve our odds dramatically. The evidence just isn't extraordinary enough — you can guess a single coin toss correctly 50% of the time with no special skills involved. It all rests on how improbable our evidence, P(B), actually is and a 50:50 chance isn't particularly improbable. For two coin tosses, P(B) becomes 0.25, and for 10 coin tosses, it comes to roughly 0.00097. Plugging those numbers into Bayes' theorem gives us a probability of genuine skill (given P(A) of a million-to-one) of around 0.0009, which although still small is a considerable improvement on that original million-to-one chance. By 20 or so correctly guessed coin tosses, the skill is starting to look a lot more genuine.
This is the basic idea underpinning statistical significance; is it more likely that our evidence is random, or due to a real effect, and is the improbability of the evidence presented in proportion to the improbability of the claim being made? But Sagan's quip about extraordinary evidence doesn't just mean that we can take someone's word for it if they managed to toss so many coins in a row. Derren Brown can pull off such a feat with some effort and misdirection as shown in his special on The System, so we always need to consider alternative hypotheses and compare how likely they are. As with Derren Brown tossing a coin with 10 heads in a row, is it more likely that they're psychic, or are cheating? So, tests such as James Randi's million-dollar challenge will control for this potential factor, making sure that the probability of foul play, fraud, and cheating is far less than the probability of genuine psychic power.
Additional note[edit]
The converse of the proposition that 'extraordinary claims require extraordinary evidence' would be that any claim requires some evidence. Therefore, to argue that a competing ordinary claim is more likely to be true than an extraordinary one, simply because the extraordinary one has no "extraordinary" evidence to support it, fails to account for the possibility that the competing "ordinary" claim has no evidence at all to support it (ordinary or otherwise).
See also[edit]
Notes[edit]
- ↑ "De ce qui précède, nous devons généralement conclure que, plus un fait est extraordinaire, plus il a besoin d'être appuyé de fortes preuves; car, ceux qui l'attestent pouvant ou tromper ou avoir été trompés, ces deux causes sont d'autant plus probables que la réalité du fait l'est moins en elle-même."[1]
- ↑ Tossed by someone else, because methods of tossing a coin may affect the outcome.
References[edit]
- ↑ Théorie analytique des probabilités by (1812). Gauthier-Villars. Œuvres Complète de Laplace: Tome Septiême. Troisième Édition (1820) . Page xvi.
- ↑ Hume, David (1748). An Enquiry concerning Human Understanding, chap. 10.4.
- ↑ Marcello Truzzi, On the Extraordinary: An Attempt at Clarification, Zetetic Scholar, Vol. 1, No. 1, p. 11, 1978.