From RationalWiki
Jump to: navigation, search
Part of a
convergent series on


This article is 78.6% referenced.
There are three kinds of lies: lies, damned lies, and statistics.
—Attributed by Mark Twain to Benjamin Disraeli

Statistics is the use of mathematics to interpret data taken from representative samples of a particular population of events or opinions to give a reasonable estimate of the whole population's outcome or opinion. Statistics is a form of inductive reasoning, but because it uses almost exclusively mathematics to make its arguments and because most statistical tests are now run automatically by modern computer software it has somehow gained the status of deductive reasoning to many. When someone feeds their data into a statistics program and it comes out significant it is treated almost like a proof (in the mathematical sense) that the alternative hypothesis is correct. A p-value is the probability of obtaining a result equal to or more extreme than what was actually observed, when the null hypothesis (H0) is true. For example, p<0.01 means that that there is a <1% chance that the null hypothesis fits the data.


[edit] The misuse of statistics

Kent Brockman: Mr. Simpson, how do you respond to the charge that petty vandalism such as grafitti is down eighty percent, while heavy sack-beatings are up a shocking nine-hundred percent?
Homer Simpson: Oh, people can come up with statistics to prove anything, Kent. Forfty percent of all people know that.
Kent Brockman: I... see.
The Simpsons, Homer the Vigilante[1]

This problem has become much greater in modern times because more data is available. If you're looking for a statistic to support your argument you can often find one, even if it's partial, selective, out-of-date or invented. It's also the case that statistical software packages are available that do all the math for researchers. This leads to the phenomenon where data goes into the black box and statistical significance comes out magically. Researchers can run a wide range of statistical test without having a clue what they are doing or what the underlying math is doing. Statistical tests are very sensitive to the structure of data and key assumptions that must be met or the results are meaningless.

It is important to always keep in mind that statistics are simply an argument and just like any other argument it does not exist in a vacuum. The reliability of its assumptions, the accuracy of its propositions and the relationship of all of these to the conclusions being drawn are all subject to as many problems as any argument made by words. Because of the overwhelming focus on "significance" being the goal a series of major endemic biases are built into most published literature (see statistical significance for this discussion) and as such single results in single experiments are worthless for creating an accepted body of knowledge. Results must be reliable and repeatable, many cranks and quacks will take advantage of the exalted status of statistics and the likelihood of finding a few "significant" results by chance alone to sell their pseudoscience and woo nonsense.

Along the same lines, it is important to remember that a statistic is simply a number. Without knowing the background information, such as sample size, alpha level, etc., it is difficult (if not impossible) to draw any real conclusions. Moreover, a statistic merely describes a relationship… it does not comment on "cause and effect" (see Causalation).

Partly in response to the replication crisis, the American Statistical Association reiterated a set of 6 principles for interpreting p-values:[2]

  1. P-values can indicate how incompatible the data are with a specified statistical model.
  2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
  3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
  4. Proper inference requires full reporting and transparency.
  5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
  6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.

Those wishing to see examples of exactly how not to use statistics should see our article Conservapedia:Schlafly Statistics.

[edit] The misunderstanding of statistics

88.2% of statistics are made up on the spot.
—Vic Reeves

A common problem is a misunderstanding of what a statistic actually means.

For example, life expectancy is often confused with maximum life span as seen in the In the Search of... episode "The Man Who Would Not Die" (About Count of St. Germain) where it is stated "Evidence recently discovered in the British Museum indicates that St. Germain may have well been the long lost third son of Rákóczi born in Transylvania in 1694. If he died in Germany in 1784, he lived 90 years. The average life expectancy in the 18th century was 35 years. Fifty was a ripe old age. Ninety... was forever." This ignores the fact that life expectancy is an average with high child mortality rates bringing that number down. In fact, Socrates, Saint Anthony, Michelangelo, and Ben Franklin all lived way past the life expectancy of their times.[3]

Another example is provided in Carl Sagan's Demon Haunted World with "President Dwight Eisenhower expressing astonishment and alarm on discovering that fully half of all Americans have below average intelligence"[4]

[edit] Topics in Statistics

[edit] Standard deviation

See the main article on this topic: Standard deviation

In statistics, a measure of how spread out the data is standard deviation.

[edit] Sample size

What is the proper sample size and how should it change opinion based samples (opinion polls) vs., static samples (number of black beans in a jar of mixed beans).

[edit] Sampling bias

Sampling bias occurs when the population being sampled is not truly representative of the population as a whole to whom the study is relevant. In such a situation the results are skewed because some members of the larger population are more likely to be sampled than others. An example of this would be a school, deciding to issue a survey to determine how funding for the semester should be allocated. On the day in question the physics students, normally forming the majority of the population, are on a field-trip to the local cardboard box factory — leading to their views being under-represented in the data. They return to find that next semester the school shall shift funding away from them and into sports.

Sampling bias isn't necessarily dishonest, but will draw into question the validity of the results. In some cases self-selection bias may occur, by which participants can choose to opt-in or out of the survey. Not all participants may be equally motivated to complete a survey, and this can make the sample set unrepresentative. This can easily occur in online polls regarding divisive issues. For example, Fox News could post a poll asking the question "Is Obama ineligible for the presidency?" The average reader may encounter the poll and provide an answer, but those with little interest in this issue may simply ignore the poll. Elsewhere a real-estate agent, dentist and alleged lawyer notices the poll and immediate directs everyone on her mailing list to go and vote for the removal of the uppity president. With sufficient following a relatively small number of birthers may succeed in giving the impression that their position is shared by a greater percentage of the larger population than is actually the case.

A way to control for sampling bias is to accurately track the demographics of those participating in a survey in order to establish a sample set that, when scaled-up, represents the population the survey is purported to address.

[edit] See also

[edit] Footnotes

  2. [ The ASA's statement on p-values: context, process, and purpose] Ronald L. Wasserstein & Nicole A. Lazar (2016) The American Statistician. DOI:10.1080/00031305.2016.1154108.
  3. Wanjek, Christopher (2002), Bad Medicine: Misconceptions and Misuses Revealed, from Distance Healing to Vitamin O, Wiley, p. 70-72, ISBN 047143499X.
  4. Sagan, Carl (1995) Demon Haunted World p. 214, ISBN 0-394-53512-X.

Mathematics Articles on RationalWiki


Conservapedian mathematics  -  Fermat's last theorem  -  Fibonacci sequence  -  Golden Ratio  -  Gödel's incompleteness theorems  -  Hypatia of Alexandria  -  Information  -  Mathematics  -  Metric system  -  Phli (fun)  -  Pyramid  -  Rene Descartes  -  Sophie Germain  -  wikiFactor  -  Zero  -
Personal tools