Reproducibility

Reproducibility is a key part of the scientific method and refers to the possibility of reproducing an experimental result by a third party following the same procedures as the original experimenter.

Having the experiment repeated by an independent investigator reduces the impact of a single study's experimental error and increases the chances of the hypothesis being accepted by the scientific community. Of course, reproducibility alone is not enough when even random numbers can reproduce prior results.

One of the signs of pseudoscience is a lack of reproducibility in their experiments. Scientific experiments can also lack reproducibility, but this just indicates either that the original experiment was flawed, that a better understanding of the problem is needed, or it is something that for its nature happens very often.

Replication crisis
The replication crisis means that many scientific experiments fail to replicate. Experiment replication attempts are part of the philosophy of good science. However, in the contemporary academic community, there are powerful disincentives against them. Academic journals are generally not interested in publishing replication attempts, and publications are important for the career of scientists. In contrast, academic journals are interested in novel experiments with significant results.

A failure to replicate an experiment does not necessarily mean that there was anything wrong with the original experiment or that questionable research practices took place; the scientific effect found in the original experiment may be real and the failed replication may be due to chance, or due to differences in methodology. Obtaining the same or similar results using different methodology does, however, increase confidence in the results.

The replication crisis is most pronounced in the field of psychology. A large-scale replication project was published in 2015 of the results of replication attempts of original published psychology experiments from 2008. Among social psychology experiments, only 14 out of 55 (25%) replicated; among cognitive psychology experiments, only 21/41 (50%) replicated. However, the replication crisis has also emerged in the biomedical field. The drug company Bayer could only replicate 20% to 25% of 67 peer-reviewed studies in oncology, women's health, and cardiovascular diseases that were performed between 2007 and 2011. Another drug company, Amgen, could only replicate 6 out of 53 "landmark" studies in cancer research.

Notably, though, the most famous "failed replication" study supposedly had its own replicability issues. A group of Harvard researchers, led by Daniel Gilbert, claimed that its failure to replicate a given study tended to coincide with divergent methodology from the original. The authors of the original 2015 study in Science responded to these criticisms by arguing that the critics' "...very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data." Gilbert et al.'s conclusions have been criticized for other reasons as well, including fundamental statistical errors.