Science's most sacred quality filter is running on unpaid labor, anonymous grudges, and a system that has never been seriously tested against the alternative of doing nothing at all.

What the textbook skips about how this actually works

Here is the official story: a scientist finishes a study, submits it to a journal, and two or three expert reviewers read it carefully, catch the errors, and either approve it or send it back for revision. Clean. Rigorous. Self-correcting. Now here is what actually happens. Reviewers are volunteers doing this work on top of full teaching and research loads, with no pay, no public credit, and no training requirement. The result, as one Michigan State University expert put it, is "a process that can feel uneven, opaque and, at times, unfair."

I remember sitting in a graduate seminar where a senior researcher described getting a rejection letter so contradictory that one reviewer praised the methodology while the other called it fundamentally flawed. Same paper. Same data. Two completely opposite verdicts. That is not a bug in an otherwise sound system. That is the system.

The deeper problem is structural. Peer review is the method by which grants are allocated, papers published, academics promoted, and Nobel prizes won — yet a systematic review of all available evidence concluded that "the practice of peer review is based on faith in its effects, rather than on facts." That quote comes from a former editor of the BMJ writing in a peer-reviewed journal, which is either deeply ironic or perfectly on brand.

The labor crisis nobody in science wants to talk about

The system is collapsing under its own weight right now. Journal editors report sending up to 35 invitations just to secure two reviewers for a single manuscript — a figure cited by editors working across Springer Nature, Frontiers, and Wiley. The responses they get back are: lack of time, not my expertise, or simply silence.

This is not a minor inconvenience. One study of biomedical journals found that in 2015, just 20% of researchers performed up to 94% of all peer reviewing. A tiny, exhausted cohort is holding up the entire edifice of scientific publishing. That is not a quality filter. That is a pressure cooker.

“The peer review system, long considered the gold standard of scientific quality control, is in crisis. Finding volunteer reviewers has become a Sisyphean task.”

— Emanuele et al., PMC / Cureus, 2025

Would you trust a food safety inspection system where the inspectors were unpaid volunteers, anonymous, and could simply decline to show up? Tell me that is a rigorous standard.

Fraud slips through and the math on that is damning

Peer review is better at identifying sloppy thinking than it is at detecting fraud. If data is fabricated or manipulated, a reviewer working under time pressure and without access to the raw data will likely miss it entirely. In recent years, a growing number of published papers have been retracted after concerns about plagiarism or faked results arose — and that trend has shaken confidence in the system.

The BMJ ran a study where editors deliberately inserted major errors into papers and sent them to many reviewers. Nobody ever spotted all of the errors. Most reviewers spotted only about a quarter. Some spotted none at all. This is the system that politicians cite when they say a finding is "peer reviewed" as if that settles the argument.

The bias problem is real and it runs deep

Single-blind review — where the reviewer knows who wrote the paper but the author does not know who is reviewing — is still the most common format in science and medicine. That means a junior researcher from a small university in a non-English-speaking country is being judged by someone who knows exactly where they come from. The Matthew Effect is real: famous names get published; unknown names get rejected. The editorial peer review process has also been strongly biased against negative studies — research that finds an intervention does not work — which directly distorts the information base of medicine.

Some journals let authors suggest their own reviewers. This feature has been misused to create peer review rings, where suggested reviewers were accomplices of the authors, or even the authors themselves using secret accounts. This is not a fringe scandal. It is a documented, recurring pattern.

The strongest defense of peer review and why it still falls short

The honest counterargument is this: peer review is not meant to be perfect. It is meant to be a minimum filter, and most researchers still trust it more than the alternative of no filter at all. A clear majority of medical researchers in international surveys say they trust peer-reviewed science despite frustrations with how slow or inconsistent the process can be. That is a real and legitimate point.

But I do not buy the "least worst system" defense when the system has never been seriously tested. One BMJ editor conducted a study where he alone decided which papers to publish, then let them go through full peer review. There was little difference between his solo picks and those selected after the full process. That is a devastating finding that the scientific community has largely chosen to ignore.

What good reform actually looks like, and who is blocking it

Open peer review — where reviewer comments are published alongside articles — is genuinely good policy. Journals like eLife and PLOS Biology already practice it, and the transparency it creates is exactly what the system needs. Accountability improves behavior. That is not a radical idea.

Paying reviewers is also overdue. The current model relies on token recognition and database access while publishers collect subscription revenue and article processing fees worth billions. Asking scientists to subsidize that with free labor while their careers depend on the outcome of the very process they are propping up is not a collegial arrangement. It is exploitation.

The political dimension here is not subtle. Governments fund science, cite peer review as the legitimacy standard for policy decisions, and then do nothing to fix the structural failures that make that standard unreliable. When a health agency says a finding is peer reviewed, the public hears "verified." What they should hear is: "passed a rushed, unpaid, anonymous check by one or two overworked experts who may or may not have read it carefully."

The system is not beyond saving. But saving it requires admitting it is broken — and that is the one thing the institutions that depend on its authority are least willing to do.

If peer review is the gold standard of science, what does it say that the gold has never been properly assayed?