File Drawer Problem


The basis of scientific advancement is based on the scientific method — essentially, stating a hypothesis, formulating a procedure for testing it, and analyzing the results.  The goal is to publish, allowing others to review and even replicate the studies.  Unfortunately, academic journals tend to only accept novel studies with significant findings (that is, where the hypothesis was supported).

This tendency, known as the file-drawer problem, because there are essentially lots of research just sitting around in a file drawer, the research is considered unpublishable but still fills in the picture of what is going on. Not having these studies published has a couple of negative side effects.  First, this means that if an effect does not appear in the literature, it becomes difficult to tell whether no evidence of it exists, or if it just hasn't been studied.

Second, most hypothesis tests have error rates between 1-10%.  The only way to know whether an observed effect is due to an error is by replicating the study.  However, with neither replicated nor non-supported hypotheses being published, it becomes difficult to determine, at best.

As a result, the file drawer problem limits the effectiveness of academic research.  Unfortunately, without significant changes to the current academic publishing model, it is unlikely to be resolved anytime soon.