Anecdotal evidence

From RationalWiki
(Difference between revisions)
Jump to: navigation, search
m (Anecdotal evidence)
Line 3: Line 3:
 
==Anecdotal evidence==
 
==Anecdotal evidence==
 
{{cquote|<nowiki>[</nowiki>[[Bloodletting]]<nowiki>]</nowiki> removes coma. Mr. Henry Clymer was suddenly relieved of this alarming symptom, in the fever of 1794, by the loss of twelve ounces of blood.|||[[Benjamin Rush]], [http://bobarnebeck.com/defence.html ''A Defence of Blood-letting, as a Remedy for Certain Diseases'']}}
 
{{cquote|<nowiki>[</nowiki>[[Bloodletting]]<nowiki>]</nowiki> removes coma. Mr. Henry Clymer was suddenly relieved of this alarming symptom, in the fever of 1794, by the loss of twelve ounces of blood.|||[[Benjamin Rush]], [http://bobarnebeck.com/defence.html ''A Defence of Blood-letting, as a Remedy for Certain Diseases'']}}
{{cquote|When a cook in [Jean] Nicot's household nearly cut off his thumb with a chopping knife, the steward ran for the tobacco plant and bound the thumb back on; after five or six dressings of the same sort, the wound healed.|||''[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1079499/ Medicinal uses of tobacco in history'']}}
+
{{cquote|When a cook in [Jean] Nicot's household nearly cut off his thumb with a chopping knife, the steward ran for the [[tobacco]] plant and bound the thumb back on; after five or six dressings of the same sort, the wound healed.|||''[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1079499/ Medicinal uses of tobacco in history'']}}
 
{{cquote|If you are basing your claims on anecdotal experience, then any treatment will seem to work for anything and everything.|||[[Steven Novella]]<ref>[http://www.sciencebasedmedicine.org/the-cam-worldview/ ''The CAM Worldview''], [[Science-Based Medicine]], October 23, 2013.</ref>}}
 
{{cquote|If you are basing your claims on anecdotal experience, then any treatment will seem to work for anything and everything.|||[[Steven Novella]]<ref>[http://www.sciencebasedmedicine.org/the-cam-worldview/ ''The CAM Worldview''], [[Science-Based Medicine]], October 23, 2013.</ref>}}
  

Revision as of 19:12, 4 November 2013

An anecdote is a brief story, often (auto)biographical and fairly interesting or unusual. Anecdotes illustrate particular aspects of a situation, sometimes going into great detail in the process. An example is the story of George Washington and the cherry tree.[1]

Contents

Anecdotal evidence

[Bloodletting] removes coma. Mr. Henry Clymer was suddenly relieved of this alarming symptom, in the fever of 1794, by the loss of twelve ounces of blood.
Benjamin Rush, A Defence of Blood-letting, as a Remedy for Certain Diseases
When a cook in [Jean] Nicot's household nearly cut off his thumb with a chopping knife, the steward ran for the tobacco plant and bound the thumb back on; after five or six dressings of the same sort, the wound healed.
Medicinal uses of tobacco in history
If you are basing your claims on anecdotal experience, then any treatment will seem to work for anything and everything.
Steven Novella[2]

In the world of pseudoscience, an anecdote is the equivalent of a peer-reviewed, double-blind, repeatable scientific experiment with consistent results. In other words, the true believers accept that just because something once seemed to work for someone — "Billy-Bob ate 3 pounds of apples a day for a year and his cancer disappeared" — then it will work for everyone, everywhere, every time! This sort of thing is termed anecdotal evidence (or, more pejoratively, anecdata[3]) and whereas it is usually the starting point of a proper scientific investigation, it is all too often the ending point and every point of a pseudoscientific investigation. The use of anecdotal evidence to draw a conclusion is like using the NBA all-star teams to estimate the average height of Americans. Try that in an introductory statistics class and you will be rewarded with a big fat F.

Anecdotal evidence is often used in politics, journalism, blogs and many other contexts to make or imply generalisations based on very limited and cherry-picked examples, rather than reliable statistical studies. A classic instance was Ronald Reagan's story of a "welfare queen" who was abusing the system, who Reagan attempted to portray as indicative of the average welfare recipient. It turned out she didn't even exist when some reporters finally decided to look for her.

Remember: the plural of "anecdote" is not "data."

Acceptability

While anecdotes — when true, at least — are nice illustrative stories, they do not constitute evidence. This is because anecdotes only ever apply to individuals or individual experiences and are subject to the biases that this brings with it. It is impossible to say that an individual anecdote is representative and it is also impossible to actually detect the real cause of the anecdote.

For instance, with life-saving medical treatments (say, pills that reduce heart-disease and subsequently lower the death rate), there are some deaths that occur whether or not the medication was taken. Therefore, if someone who is on the medication dies, you cannot tell if they would have died anyway without it — you can't prove that the medical intervention worked, or not, from the one case study. If someone with a rash gets better after taking a homeopathic remedy, that doesn't mean the remedy caused the improvement, as it could simply have been an exception to the rule. The anecdote doesn't tell you anything about all the people who did not get better; many other people with rashes could have taken homeopathic remedies to no effect.

Or take cancer. Let's assume 1 million people (1,000,000) decide to take a homeopathic remedy to cure their cancer instead of taking real medicine. Let's further assume, to be generous, that 0.1% of this million will experience spontaneous remission (the actual remission rate, for breast cancer and basal cell carcinoma at least, is closer to 20%[4][5]), and that 0.3% were misdiagnosed and so do not actually have cancer. This makes for a total of 0.4%, or 4000 people. Now, 4000 people translates into a lot of testimonials, "CAM cured my cancer" blogs, Internet comments, and real-life word-of-mouth, so this makes an extremely positive impression for homeopathy. But the other 99.6% died, so though you are bombarded with the stories of the 4000, you hear absolutely nothing about the 996,000 who were not cured.

There are also anecdotes in which a person was sick for many years, and no conventional treatment was able to cure them, but they got better after applying some alternative remedy for a few days/weeks/months (Nicolas Monardes used such stories to promote medicinal tobacco use[6]). The implication is that the improvement cannot possibly have been a coincidence, because it was so unlikely. Of course, since thousands (if not millions) of people use alternative medicines, such an event is almost certain happen to someone, even if the odds of it happening in one specific instance is low. The odds of a particular person winning the lottery is extremely low, but somebody always does, regardless. Not to mention that most people get sick many times (probably over 100) in the course of their lifetime, and some improbable coincidence is bound to happen during at least one, if not several, of these illnesses.

While "case studies" exist within medicine that are essentially anecdotal, these are treated mostly as curiosities or communications between doctors, rather than hard evidence on par with real trials.

Selective reporting is the main factor that popularly disseminated anecdotes will be almost certainly unrepresentative. People will tend to only ever report the interesting or the unusual or the powerfully emotive events but never the mundane. After all, no one ever sold a newspaper where the front-page headline read "girl receives new vaccine; suffers zero complications and is now alive and well". Regression to the mean is also a problem.

The power of anecdotes

Anecdotal evidence is not accepted as proper proof or scientific evidence for good reason, yet they still dominate people's thinking. This is because while facts and figures may be true, they are difficult for people to relate to. Anecdotes have convincing and strong narratives that resonate with people on a key individual level — someone telling an anecdote will lead the listener to think that it just could happen to them too. There are further biases at work. Anecdotes, even implausible ones, can contain a high level of detail. It is this detail that causes people to become involved in the story and to imagine it more clearly; a story about a young girl won't be nearly as widely spread and accepted as a story about a five year-old girl called Sally from Wisconsin who was cycling to school one day.

The biases at work in causing people to add disproportionate weight to anecdotes cause major problems during health scares. Particularly with vaccine hysteria where it is a case of hard, scientific and proven (yet dry and difficult to grasp) evidence versus strong narrative stories told by concerned parents. It is often advised in such circumstances that those on the side of science should also use their own anecdotes to take advantage of their persuasive power; e.g., instead of giving statistics of vaccination uptake dropping and illnesses increasing, tell — in great detail — stories about specific deaths that have occurred from vaccine preventable illnesses. Respectable scientists, however, may be against such manipulative tactics.

Support in favor of anecdotal evidence

My grandma heeded anecdotal evidence and she lived to 106.

Also, Jeanne Calment, the person with the longest confirmed lifespan (122 years) smoked for almost her entire adult life and nothing bad happened to her.[7][8] By alternative medicine logic, this should be indisputable proof that tobacco is actually good for you. Because, you know, there's no such thing as flukes. Flukes are a myth invented by allopaths.

See also

External links

Footnotes

  1. This anecdote is now almost universally considered to be untrue. See Washington's Cherry Tree - Legend or Fact?.
  2. The CAM Worldview, Science-Based Medicine, October 23, 2013.
  3. Anecdata entry at WordSpy.
  4. Spontaneous Regression of Melanoma May Offer Insight Into Cancer Immunology, Carrie Printz, JNCI J Natl Cancer Inst (2001) 93 (14): 1047-1048.
  5. Study reveals surprising cancer remission rate, Dartmouth Medicine Magazine.
  6. Of the Tabaco and of His Greate Vertues
  7. http://extremelongevity.net/supercentenarians/worlds-top-ten-oldest-people-ever/oldest-person-ever-jeanne-calment/
  8. http://www.endlesshumanpotential.com/jeanne-calment.html
Personal tools
Namespaces

Variants
Actions
Navigation
Community
Tools
support