When the Reviewer Isn’t Human: AI and the future of scientific judgment (#442)

Artificial intelligence has entered the world of scientific publishing with astonishing speed.

What began as a convenience for grammar correction and language polishing has evolved into something far more potent: an analytical assistant, a reference engine, and, increasingly, a silent reviewer.

The appearance of AI-generated text in manuscripts, and even AI-assisted peer reviews, has raised fundamental questions.

What happens when artificial intelligence becomes not just a tool, but a final arbiter of scientific merit?

And what are the long-term consequences of taking the human factor out of the equation?

The Seductive Efficiency of the Machine

The appeal of AI in science is undeniable.

Large language models can rapidly identify grammatical errors, highlight missing controls, suggest references, and even detect inconsistencies between a study’s abstract, data tables, and conclusions. They are tireless, objective, and fast. They don’t get bored, biased, or overburdened.

For editors struggling to find reviewers and for authors writing in a second language, this efficiency feels like salvation.

AI promises speed, fairness, and standardization.

A seductive trinity for an overtaxed scientific community.

But the very qualities that make AI efficient also make it soulless.

Peer review has never been simply about technical correctness. It is about judgment. It is a conversation between minds, an act of mentorship, and an expression of the moral and intellectual life of a discipline.

A good reviewer doesn’t merely check whether the data support the conclusion. They sense the courage of a new idea, the craftsmanship of an elegant experiment, and the quiet beauty of restraint.

AI, for all its power, cannot feel these things.

AI can rank, summarize, and emulate, but it cannot care.

The Risk of Removing the Human Element

When AI shifts from assistant to arbiter, subtle but dangerous effects follow. The culture of science risks losing its interpretive depth and moral compass.

AI tends to favor conformity. The statistically sound but unimaginative. The safe over the bold.

A manuscript that bends convention or ventures into unorthodox territory may be downgraded not because it is wrong, but because it is different.

If peer review becomes algorithmic, science risks flattening into homogeneity.

Technically impeccable papers that lack curiosity, risk, or wonder. In such a world, scientific creativity becomes collateral damage in the pursuit of efficiency.

Peer Review as a Hidden Classroom

For generations, peer review has served as an invisible apprenticeship. Junior researchers learned critical thinking not just by writing, but by reviewing—and by watching how experienced scientists wrestled with uncertainty and nuance.

That dialogue is the transmission of a profession’s culture. If we replace it with machine evaluation, we lose mentorship, conversation, and accountability.

Future scientists may learn how to submit papers, but not how to question them.

The art of judgment, already in short supply, may atrophy entirely.

The dumbing down of review is not about ignorance. It’s about abdication. It’s about outsourcing thought to a tool that cannot understand the meaning of what it reads.

Authors, Reviewers, and the Boundaries of AI

The ethics of AI use differ sharply between authors and reviewers.

• For authors

• AI can be a legitimate assistant, helping with grammar, structure, and clarity, so long as the human writer retains full accountability for the ideas and accuracy.

• Many publishers now require authors to disclose any AI assistance, emphasizing that the human author must remain responsible for the integrity and originality of the work.

• For reviewers

• The landscape is more perilous.

• A manuscript under review is confidential intellectual property.

• Uploading it into a public AI platform may inadvertently expose it to external servers, violating confidentiality and possibly contributing to model training data.

  
  

The editors of JAMA and the JAMA Network recently codified this risk in their updated guidance:

Reviewers are prohibited from uploading confidential manuscripts into AI systems and must disclose if they use AI as a resource during review.

The principle is simple but profound. Confidentiality is not optional, and accountability cannot be delegated.

Policies in Transition

Major editorial bodies, including the International Committee of Medical Journal Editors (ICMJE), have echoed this principle. They emphasize that AI cannot be credited as an author because it cannot be held accountable for ethical judgment or integrity.

The current trajectory suggests a human-in-the-loop model:

AI may assist with identifying checklist items, summarizing manuscripts, and comparing data for consistency, but decisions must remain human.

Even advocates of AI in peer review liken its role to driver-assistance technology. Useful for monitoring blind spots, but not for taking hands off the wheel. The vision is not automation, but augmentation. A collaborative workflow in which AI-generated summaries support, but never replace, human insight.

The Data Question: Does AI Keep What It Sees?

One of the least understood aspects of AI,and many other AI-adjacent tools in publishing, is how they handle data.

In consumer versions of ChatGPT, user inputs may be retained for model training unless data sharing is explicitly disabled. Enterprise or API versions, by contrast, claim not to use input data for training.

Thus, when reviewers paste manuscript content into a personal account, there is a real risk, however small, that the text becomes part of a training corpus.

But a parallel hazard exists in an area many editors take for granted. Anti-plagiarism software.

Journals increasingly require authors to run manuscripts through similarity-checking systems. Some of these store text on external servers, meaning that unpublished data, figures, or study results can become partially accessible online or appear in future cross-check databases.

In both cases, AI and plagiarism detection, the same rule applies:

Transparency, consent, and data control are essential.

Confidentiality ends not when a file is uploaded, but when it is entrusted to a system whose architecture the user does not fully understand.

The Unresolved Risks: Bias, Hallucination, and Cognitive Offloading

Even within secure systems, the risks are not purely technical. As the JAMA editorial warns,

AI-generated reviews can include fabricated citations and factual confabulations, often written in authoritative language that masks error. Checking such errors can increase human workload rather than reduce it.

Bias is another concern. Studies show that language models may favor certain topics, writing styles, or even degrees of optimism. They lack jealousy, vendettas, or career motives, but they do exhibit subtle algorithmic preferences, including favoring their own phrasing over human prose.

Equally troubling is what cognitive psychologists call cognitive offloading.

Reviewers relying too heavily on AI may think less deeply. The danger is not just inaccuracy, it is intellectual atrophy.

Peer review’s greatest contributions often come from flashes of human intuition. The perceptive question, the unexpected connection, the creative doubt. No machine can replicate that spark.

Rick’s Commentary

AI can undoubtedly enhance aspects of peer review. It can flag missing statistical details, summarize long manuscripts, or cross-check internal consistency. These are welcome efficiencies.

But AI must remain a servant, not a sovereign.

Science depends not only on data but on discernment. On people willing to interpret, challenge, and care.

Reviewers may use AI to improve their own prose, but not to evaluate someone else’s confidential work.

Authors may disclose AI assistance, but they must own their words and their reasoning.

Editors, meanwhile, must provide evolving guidance so innovation never outpaces ethics.

At its heart, science is not a database of facts.

Science is a conversation about truth.

That conversation depends on curiosity, empathy, and moral courage. These are uniquely human qualities that algorithms cannot emulate.

If we remove the human factor, we risk turning science into a monologue. Efficient. Uniform. And hollow.

AI may one day become the perfect reviewer, but it will always be an imperfect scientist.

The value of peer review, and indeed of science itself, lies not only in what we know, but in how we come to know it.

And that process, at its best, remains profoundly human.

Further Reading

Artificial Intelligence in Peer Review. https://jamanetwork.com/journals/jama/fullarticle/2838453

Toward responsible use of artificial intelligence in our journals. https://avmajournals.avma.org/view/journals/javma/263/11/javma.263.11.1342.xml

#bunyipsbooksbeyond