Talk:Extraordinary claims require extraordinary evidence

	This Assessing pseudoscience related article has not received a brainstar for quality. Please consider expanding the article appropriately. See RationalWiki:Article rating for more information.
Editorial notes needs references

I consider myself a confirmed skeptic, but I'm not sure about the "extraordinary claims require . . ." rule. The Alice and Bob analogy is clever, but what happens when we try to get a bit closer to real life? There are lots of phenomena which we accept as proven (understanding that "proven" in science doesn't mean that it's above questioning) even though we don't understand why they exist, for instance many drugs have been proven by accepted means (double-blind studies, controls, statistical analysis, etc.) even though we don't understand their mechanism of action. If these same tests gave positive results for some presumably pseudoscientific practice, e.g., dowsing, remote healing, homeopathy, etc., should we require greater evidence than we would for more "scientific" drug trials?

I realize there is a bit of a difference -- in the standard drug trial example, while we don't know the exact mechanism of the drug's action we know that in general that chemicals can have biological effects, whereas we can't say the same for, say, dowsing or diluting a substance to the point where there's nothing there -- but is that enough to require "extraordinary evidence?" (The homeopathy article touches on this as well.) If we believe in the scientific method, why do we trust that method for some things and not for others?

Just wondering . . . ^{— Unsigned, by: Igor4458 / talk / contribs}

First off you have to separate out the difference between trying to codify an observable phenomenon and trying to explain the theory behind it. As you show in your example these can be easily separated out. A lot of the problem with pseudoscience, quack medicine, and woo is that they are trying to push a theory not just something that could be observed.

For example, lets say we take something like electronic voice phenomenon. If the question is can you hear a sentence in the white noise of the tape, the answer is obviously yes. This is a repeatable, verifiable phenomenon. Where things get iffy is when the paranormalist claims that it is the voice of a ghost. If I offered the alternative explanation that it is merely a product of pattern recognition in the human brain, which claim would you require more evidence for? My claim fits well with in the current accepted and understood paradigm of psychology, reality and human understanding. The other answer is "extraordinarily" outside the realm of accepted knowledge. I would think to accept the ghost story would require a bit more evidence.

In your drug example, if someone claimed that the drug worked because it tapped into a spiritual funnel next to Neptune and recharged our auras, I would think you would want more evidence compared to the explanation that the drug work through a biochemical interaction in the body that we have not yet fully elucidated.

Another point, data that is drawn from a random distribution can show extreme effects by chance a lone. We set how extreme an effect we are willing to call "significant" when doing drug studies. The standard is often something like 5 percent. Basically, there is a 5 percent chance to show the "effect" of the drug by random chance alone. If 19 studies in a row show the drug had zero effect, but the 20th study (a solid study all around) showed an effect, would you believe that one study? What if it was a 1000 studies of no effect and 5 with an effect? The point is, for something like homeopathy, there is so much data out there that has shown no effect and the few studies that do show an effect our fraught with problems, that it would take a lot of evidence to swing things around.

So a claim that something is effective when there is a ton of evidence that it is not is an "extraordinary claim" and will require a lot of evidence in order to overcome what we all ready know. tmtoulouse 23:45, 12 November 2009 (UTC)

(EC, in reply to Igor) Interesting points. Thinking about this, I reckon that the key is that a valid scientific hypothesis should have some basis or at least parallels in established scientific knowledge. In the drug test example, we know that there are thousands of proven cases where a specific chemical has a specific effect on the body, so finding, testing and proving a new and different example of this phenomenon is standard scientific practice. Even if we might not yet know the exact mechanism in each case, we can easily see the parallels with myriad other cases. On the other hand, telepathy, as an example, does not closely relate to any proven scientific phenomenon. That's what makes it an extraordinary claim. If it could be proven with certainty that some people are genuinely telepathic, it would open up a whole new untapped area for investigation. But to prove this would require a very heavy burden of proof as well as well as some kind of coherent scientific hypothesis to explain the phenomenon (I'm using that word too much). It's rational to doubt claims which have no firm foothold in what is already known. Wẽãšẽĩõĩď^{Methinks it is a Weasel} 23:58, 12 November 2009 (UTC)

All helpful comments, but incomplete. Tmtoulouse says "In your [meaning my] drug example, if someone claimed that the drug worked because it tapped into a spiritual funnel next to Neptune and recharged our auras, I would think you would want more evidence compared to the explanation that the drug work through a biochemical interaction in the body that we have not yet fully elucidated." Actually, I'm not sure that I'd care a lot about someone's conjecture about how it works if it in fact works. Isn't how separate from whether it works? So to continue with the drug/homeopathy example (not that it should matter to the discussion, but I want to make clear that I think homeopathy is a complete fraud), if tests of efficacy of a homeopathic treatment showed positive results similar to results of a trial of a more "normal" drug -- same 95% (or whatever cut-off is applied) confidence level, same number of tests, same results, etc. -- why would something more be required? Is it simply because we "know" that the homeopathic stuff is just distilled water while the other drug (OK, maybe I shouldn't say "other") is, well, a drug, even though we have no idea of how it acts on a particular malady? If someone claimed that the Neptunian auras were the basis for the drug's, and not the homeopathic preparation's, apparent activity, would we on that basis change our evaluation of the drug? The "extraordinary claims require . . ." principle always sounded good to me until I started thinking about how it would be applied, and then it breaks down. It starts with a subjective decision about whether a particular claim is "extraordinary," and if someone decides that it is, then we apply a higher (and how much higher also is subjective) burden of proof. Fuzzy times fuzzy doesn't sound very scientific to me. My gut reacts the same as yours in deciding what's plausible and what's hocus-pocus, but that doesn't make it, well, rational. Igor4458

Extraordinary ambiguity

Do extraordinary claims require extraordinary evidence? I'm not convinced they do. But I'm also a bit confused as to what atheists mean when they say extraordinary claims. What constitutes as an extraordinary claim? Perhaps a good definition of an extraordinary claim is any claim which if true would make a change in at least one of our beliefs. If this is the definition of an extraordinary claim, then it seems to me the proponent of this view has a serious problem. That being, how do you produce enough evidence to support the extraordinary claim that every single extraordinary claim requires extraordinary evidence? It seems to me if this statement were true it would have a significant impact on all of our beliefs and more than the atheist perhaps would like to admit. For example; what extraordinary evidence do we have for the quite extraordinary claim that other minds exist? It seems to me the only evidence we have for such a proposition is that we perceive other conscious beings. But this hardly seems extraordinary, as it merely begs the question how do we know our perceptions are accurate? Now two objections might arise from this argument; first and foremost would be the complaint that we need to properly define extraordinary evidence. Well ok lets define it. I think a perfectly acceptable definition of extraordinary evidence is evidence such that it raises the probability of a proposition being true to more than 50% percent. Does the fact that we perceive other minds raise the probability that other minds exist to over 50%? I don't see how it raises the probability even one percent, the two facts are virtually irrelevant because we have no evidence that our senses are reliable, nor could we for that matter. A second objection might be that I haven't defined extraordinary claims properly. To that objection I would simply ask; what about my definition is deficient? Therefore barring other objections, it seems that if this view of rational justification is true, then we aren't justified in believing anything.

I also think that if the claim of extraordinary claims requires extraordinary evidence is true then it should not be accepted, because we don't have very good evidence for the claim. To show this lets look at the evidence used to support it. An example of a claim is usually given, for our purposes lets say someone makes the claim that the found a leprechaun and a pot of gold at the end of the rainbow. Surely this requires extraordinary evidence. I would tend to agree, it does in order to be rationally believed, but I don't think its because its an extraordinary claim (although it is), but because it doesn't have any evidence for it and it isn't apparent to our senses. See we are rational to believe in the external world and other minds, because for one those things are apparent to us, and secondly there is no way we could ever have evidence which proves or disproves either claim. Whatever evidence we could have for the existence of the external world as we observe it would have to go through our senses, but the reliability of our senses is precisely what is being questioned. Claims such as whether there are leprechauns in the world we perceive seem falsifiable and therefore require extraordinary evidence, such as seeing the leprechaun and the gold yourself. But there is another question which needs to be asked here; How do we know our perception of the leprechaun isn't a delusion? Well we don't, so perceiving it wouldn't raise it probability really at all, but it would make us rational in believing in leprechauns, and for the same reasons as before. So as we have seen the claim that extraordinary claims requires extraordinary evidence, does not have adequate support for it and thus should not be accepted.

But what does this imply for the question of God's existence? Well It seems to me that the claim that God exists would be extraordinary but I'm not so sure you need to have extraordinary evidence to believe in God rationally. After all I seriously doubt we could provide extraordinary evidence for most things we believe rationally. But I do think we have evidence for God's existence, there are many arguments for God's existence. But I would like to make not a reason based argument for God's existence, but an argument based in our intuitions. It seems apparent to me, when I look at the grandeur of the universe, my immediate reaction is that there is something divine which created it, not because it looks designed, but for similar reasons that my immediate reaction to murder is disgust. One possible objection to this is that there are immediate reactions that are false, such as our seemingly inherent belief that the heavier the object the faster it falls. The issue in this objection is that such instances are results in errors of observations, but we have no observations of universes being created nor do we have any observational basis for reacting to murder with disgust. These intuitions, it seems to me are more basic than our perceptions. So why should I not believe them? How am I irrational in trusting my intuitions? One other objection might be, why choose one religion over another? How does one choose which religion to believe in? Well I think there are a couple ways to analyze religions, but the most obvious one is the test of logic. Is the religion logically consistent? Are there contradictions in the religion? After my analysis of world religions I found Christianity to be the only completely logically consistent religion, but others might come to other conclusions, and they would still be rational, I would just believe they were mistaken. There are other questions which need to be asked such as why should we believe only what is rational? But those I shall leave for another time.

Religiosophizer149 (talk) 04:22, 29 March 2012 (UTC)

Truzzi reversed his belief in this as he wondered what 'extraordinary evidence' meant and came to believe there is no such thing as 'extraordinary' evidence, there is just ordinary evidence. Truzzi never argued this, but I think a way to look at this is with the arguments of some global warming deniers. In the recent flooding in the Ottawa Valley, a stupid meme came out saying "was there global warming that caused the flood in the Ottawa Valley in 1887?" The implicit claim here seems to be that if global warming/climate change were real it would produce weather events that had never occurred before, or, it would produce 'extraordinary things.' But, that's absurd. Global warming/climate change should only be expected to produce ordinary things like floods but in increasing numbers and severity, and that's exactly what we are now seeing. Of course, the reality of global warming/climate change is not an extraordinary claim, but I think this shows the principle that there is only 'ordinary evidence.'

What may be a fair comment about extraordinary claims is that they require extra ordinary evidence, or more evidence than a non extraordinary claim to be accepted. This extra ordinary evidence should preferably be in the form of coming from different perspectives so as to show a convergence of independent evidence. ^{— Unsigned, by: 24.87.92.171 / talk / contribs}

An Attempt at Defining "Extraordinary Claims" - and "Extraordinary Evidence"

Let us examine what can possibly be meant by an 'extraordinary claim'.

• First of all, we could consider it to mean a statement, which, if true would cause us as a person, or as a community to overturn previously held beliefs and understandings in favour of a new paradigm based on the correctness of the claim. Now that claim is 'extraordinary', because it is not like an "ordinary claim". If I claim that my house is warm in winter, then you might think that is not extraordinary, since you know I have central heating. If I claim my house is cold in winter, then again, you might think that is not extraordinary, because my central heating may have broken, or it's not functioning properly. On the other hand, if I claimed that the interior of my house was so hot it could melt lead, that would be extraordinary, and you would be right to demand evidence for it. To accept my claim that my house could melt lead, this would cause you to change your understanding of what - or where my house was. I might live in a Steel Factory, or my house might have been in Iceland, and has been swallowed up by a volcanic lava flow. If I claimed that the inside of my house was at a temperature below Absolute Zero, then the claim becomes so startling that you would not just require some evidence, but send for two doctors to test my sanity - or contact the Nobel Prize committee.
• The second implication is that the statement should assert that something is 100% TRUE (always TRUE), or 100% FALSE (always FALSE),and not leave any room for get out clauses. This means that it should not contain ambiguities or vaguenessess so as to allow goalpost-shifting. "Nothing can travel faster than the speed of light" is an extraordinary claim, since it says something can NEVER happen. "All politicians are Liars" is similarly an extraordinary claim, but one which may be refuted by pointing to George Washington (allegedly). A statement such as "Water is a poor conductor of electricity" would not qualify, because even though it is paradigm-shifting if true, it is imprecise, since we don't really know what "poor" means, and so the writer may mean, for example that water is a poorer conductor than gold - or in comparison to another form of water such as iceVI, then water is poor. A statement that something exists (or does not exist) in the real world must qualify as an extraordinary claim (providing that it is paradigm-shifting - and that no one knew of, or considered the possibility of its existence previously). The statement that The Higgs Boson exists or Santa Clause exists (ie in reality, not in fiction or in our minds) must surely count as extraordinary claims. On the other hand the claim that quarks 'might possibly' have a substructure is not extraordinary, simply because it is offering a possibility that they have a substructure, rather than ruling out entirely the possibility that they are a single, indivisible entity. If a paper in a journal claimed "All quarks have a substructure", then that shifts the claim so that it becomes extraordinary, and at that point, we start to require some pretty strong evidence to convince us
• An alternative version of the second implication could be that an extraordinary claim might seeks seek to put limits on what we can ever know, or what we can ever do. We can never travel faster than the speed of light, so it is an extraordinary claim. We can never lower the temperature of an object below absolute zero. No matter how hard you try, you will never be able to know and object's momentum and its location simultaneously. These all impose limitations on us, and so are extraordinary claims (or were, when they were first proposed).

Now let us examine what can possibly be meant by an 'extraordinary evidence'.

The second part of the statement is "extraordinary evidence", and to examine what that might be. The basic criterion needs to follow the logic above, in that if previously held beliefs are to be overturned, and replace it with the new claim, then we need to be convinced that:

• The contents of claim are exactly what is on the label; i.e. that the claim it is not intended as a metaphor, or analogy, or simply a statement of the person's own beliefs; the claim needs to be phrased in such a manner that the meaning cannot shift when challenges are made
• there is enough evidence consistent with the claim in order to establish that it is reasonable
• there have been rigorous attempts to falsify the claim which have not succeeded in finding any evidence to falsify it
• the claim has internal logical consistency, and asserting the claim is true does not lead to the contradiction of one of its own assumptions, or the mutual contradiction of two of its deductions.
• when the logical implications of the claim are thought through, they are either consistent with what we know, or if not, provide us with alternative viewpoints to replace this knowledge.
• the claim provides us with a coherent explanation of why it is true.
• the claim provides us with an understanding of why our previously held belief was false, and why this new claim needs to supersede it.

I have done this off the top of my head, but I cannot believe someone has not written about this before. I think this issue is not just about proving God exists or not, but is a set of criteria which can usefully be applied to any scientific (or pseudoscientific statement).

The application to this argument to the existence or non-existence of God

The issue of whether "God exists" is an "extraordinary claim" is interesting. The statement is not in itself paradigm-shifting, since many people already adhere to this view. In addition, one might argue, that depending upon who makes the claim, the notion of "god", may be imprecise, and the notion of existence might refer simply to the existence of god as a mental construct -individually or collectively. It also needs to be contextualised. A priest claiming that "God exists" from a church pulpit is an entirely unremarkable claim. His audience would expect him to believe that, and it is expected that most of the audience take that world view. It is certainly not paradigm-shifting. Even if the speaker were to make the same claim to an atheist convention, this still might not be considered as an extraordinary claim, if this were taken simply as an expression of the speaker's beliefs. It only becomes extraordinary if the speaker is seeking to convince the audience to accept his viewpoint as literally true, and to change their worldview as a result of the evidence he is about to present.

On the other hand, the claim that "God does not exist" does seem to be an "extraordinary claim", especially if interpreted as "The collection of all things that exist as independent entities in all possible versions of reality, does not includes the God of the monotheists". When put in such terms, it becomes an extraordinary claim, because it makes a bold assertive statement about not just what we know, but puts limits on what intelligent beings will ever discover, even if they could search forever. In addition, It is not a 'belief' in the sense that many believers "know God exists for them", ot that non-believers "have no knowledge of God", it extends this principle to a statement about the structure of the real world which extends far outside the boundaries of what we can possibly experience directly.

As a committed skeptic, I am always wary of anything which seeks to make definitive claims of the 100% true/false variety. I think that if this argument is used too often to challenge theists, it might just come back and bite atheists in the rear.

CatWatcher (talk) 11:31, 15 October 2012 (UTC)

Problem with the maths?

There may or may not be a problem with this article's use of Bayes' theorem: http://math.stackexchange.com/questions/894476/rationalwiki-on-extraordinary-claims-require-extraordinary-evidence --ZooGuard (talk) 09:29, 13 August 2014 (UTC)

Converse?...

The "Additional Note" section of the article reads: "The converse of the proposition that 'extraordinary claims require extraordinary evidence' would be that any claim requires some evidence."

While it is true that "any claim requires some evidence," it's not really the logical converse of "extraordinary claims require extraordinary evidence."

The logical converse would be "all claims which require extraordinary evidence are extraordinary claims." ^{— Unsigned, by: 46.87.48.190 / talk / contribs}

Possible rename or redirect

I've noticed that the Sagan's quote is commonly referred to as the Sagan Standard, I've been wondering whether the name of the article should be change to that or whether we should have a redirect to this.--WMS (talk) 20:04, 13 November 2020 (UTC)

Good idea, I have added a redirect. Bongolian (talk) 20:07, 13 November 2020 (UTC)

Nice one, thank you.--WMS (talk) 18:48, 14 November 2020 (UTC)