Information icon.svg Consider taking the RationalWiki Community Survey 2017 or see the results.

Sample size

From RationalWiki
Jump to: navigation, search

In statistics, a sample size is the number of observations in a statistical sample.

The larger the sample size, the more precise the estimate is of the population being studied in the statistical sample. Therefore, any study involving statistics should use a large enough sample size to draw a reasonable conclusion. More specifically, a sample should be large enough to reject the null hypothesis when, in truth, it should be rejected (i.e. statistical power). This ensures that what you observe in your study is actually real and not just a fluke.

Large samples[edit]

Large samples are particularly important in medical trials. This not only ensures that the medical treatment is effective for everyone, but it allows rare side-effects to be identified. It would be very easy to show something was safe with a small sample size of 100, only to give it to a population of millions to discover one in a thousand died. Hence samples are large and varied, importantly covering all the demographics needed, age, gender and/or lifestyle for example.

In the off chance a negative result were not suppressed, a large sample would assure that the negative finding has some merit. Without a sufficiently large sample, a study can fail to reject the null hypothesis even if the null hypothesis were false.

Small samples[edit]

Many pseudosciences rely instead on anecdotal evidence to draw their conclusions. As an anecdote involves a sample size of 1, it cannot be considered conclusive. Furthermore, multiple anecdotes do not constitute a larger sample size, but instead constitute several (often selectively chosen for their results) samples, each with a sample size of 1. Therefore, arguments based solely on anecdotes and testimony need to be studied further, but not taken as conclusive in themselves. It's often very easy to get dramatic results out of small samples because of the ease of which small numbers can, by pure chance alone, buck a real trend and give you the opposite result. If these are picked up by the media, they can start panics over what in reality is nothing. The claim that the MMR vaccine caused autism, which was rife in the UK press from 2001 to around 2005 relied solely on a scientific paper with a sample size of 12. In contrast, one of the more recent studies showing that the vaccine was safe had a sample size of 4000 (which the authors even noted was a bit on the small side).

Statistical power[edit]

Statistical power is the degree to which a study will properly reject the null hypothesis or, alternately, have a reasonably narrow confidence interval. Statistical power is the other side of statistical significance, focusing on the probability of the null hypothesis being rejected when, in fact, it is false. While conventional statistical significance is quite commonly set at 0.05 (5% chance of incorrectly rejecting the null), conventional statistical power ranges further. At a minimum, power of 0.8 (80% chance of rejecting the null when it is false) is considered adequate. Higher power, lower statistical significance levels, and smaller effect sizes all require larger sample sizes.

See also[edit]