Peer Review: Origins, Evolution, and Critiques

Core Idea

Peer review — evaluation of research by other scientists before publication — emerged gradually as the mechanism for quality control in science. Early scientific societies like the Royal Society relied on editors to judge papers; by the 20th century, peer review became standard. Peer review has advantages: it reduces publication of clearly erroneous work; it provides feedback for improvement; it creates community ownership of standards. Yet it has limitations: peers may reject novel ideas; slow review delays publication; anonymity can enable unfairness; reviewers often lack incentive to do thorough review. The crisis in replicability in some fields (psychology, medicine) has raised questions about whether peer review adequately filters poor science. P-hacking — manipulating statistical analyses to produce significant results — can escape peer review. Yet critics also recognize that alternative systems (publication without review, or publication with open post-hoc commentary) have their own problems. Peer review remains the foundation of scientific credibility, yet its shortcomings are increasingly recognized. Understanding the history and limits of peer review is important for understanding modern science: it reveals that gatekeeping is imperfect, that scientific consensus can incorporate errors, and that fixing problems requires structural changes, not just individual moral reform.

Explainer

Peer review -- the evaluation of scientific manuscripts by other researchers before publication -- is widely treated as a defining feature of science itself, a mechanism ensuring that published claims have withstood critical scrutiny. Yet peer review in its modern form is a mid-20th century innovation, and its limitations have become increasingly apparent through systematic study of scientific practice.

The earliest scientific journals -- the Royal Society's Philosophical Transactions (1665), the Journal des scavans (1665) -- relied on editorial judgment rather than external referees. Scientific societies served a gatekeeping function through membership and presentation norms, but the idea of systematically sending manuscripts to expert referees was slow to develop. Major journals adopted it at different times: Nature formalized referee review only in the 1970s. The assumption that peer review is ancient and universal is itself a myth.

The modern peer review system became established alongside the postwar expansion of scientific publishing. With thousands of journals and millions of papers annually, editors needed ways to filter submissions -- external reviewers provided that filter. Grant review at funding agencies (NSF, NIH) adopted similar procedures: panels of scientists evaluating proposals before funding decisions. Peer review thus became embedded in two of science's most important gatekeeping functions: publication and funding.

The limitations of peer review became visible through the replication crisis, which emerged most prominently in psychology and medicine in the 2000s-2010s. The 2015 Open Science Collaboration reproduced only 36-39% of 100 published psychology findings. Large-scale medicine analyses found that many standard treatments had never survived rigorous replication. The mechanism was not primarily fraud but structural: publication bias (journals prefer positive results), small sample sizes that permit chance findings, and p-hacking -- flexible analysis choices that allow researchers to cross the p < 0.05 threshold. None of these are detectable by standard peer review.

High-profile fraud cases (Hwang Woo-suk in stem cells, Diederik Stapel in psychology, Jan Hendrik Schon in physics) exposed another limitation: reviewers see manuscripts, not data, and cannot detect fabrication without access to original files.

The reform movement emerging from the replication crisis includes pre-registration (researchers publicly commit to hypotheses and analysis plans before data collection), open data requirements, registered reports (journals commit to publish regardless of results), and open peer review. These represent real structural changes to scientific practice -- acknowledgments that peer review alone is insufficient and that incentive structures rewarding novelty over replication need redesign. The history of peer review reveals not that science is broken but that its quality-control mechanisms are historically contingent and require ongoing revision.