New Delhi: The number of research papers being churned out by academia has risen dramatically over the past few years, but researchers are warning that using AI to replace human peer review could be dangerous.
A team from Stanford University and Bocconi University studied the impact of AI peer review and presented their findings on 8 July at the International Conference on Machine Learning in Seoul, South Korea. The study, titled “Stop Automating Peer Review Without Rigorous Evaluation”, was published on arXiv.
One of the findings of the study is that when Large Language Models (LLMs) are used to produce scientific peer reviews they cut down the diversity of opinions among reviewers. They also allow the process of publishing a research paper to be “gamed” in such a way that the AI will be more likely to approve it.
“They can be gamed to improve scores through fully automated paper rewriting. We call this paper laundering: cosmetic paper rewrites to increase AI review scores without improving the scientific substance,” reads the study.
The paper argues that amid the flood of research papers and the need to speed up the process of peer review, an important fact is being overlooked. Scientific peer review is the backbone of academia, it determines what research gets published, funded, and recognised. The paper defines this as a “high stakes” process and says that rigorous evaluation cannot be compromised.
Also read: Brown professor suspected mass AI cheating on his midterm. The in-person final proved it
Gaming the AI hivemind
The authors found that AI reviewers gave similar reviews when compared to humans. Common phrases were also repeated across papers suggesting that the AI review might be generic across different subjects rather than specific to a paper.
“Disagreement, though, among the perspectives of diverse human experts is an important feature of peer review, which is why the work of senior committee members in aggregating those views and collectively making final acceptance decisions is so important,” the paper read.
In addition, researchers found that AI reviewers had a preference for a certain style when it came to research papers. This meant that an author could use another LLM to reformat their research into a language which the AI reviewer might prefer and give a higher score to.
“If paper laundering becomes widespread, scientific writing will converge toward whatever style the AI reviewer rewards, risking an intellectual monoculture and discouraging diverse ways of presenting ideas,” the study said.
Researchers argue that before AI is allowed to influence what kind of research gets published scientists must find a way to make sure that human expertise is not discarded in the process.
“An effective solution requires validated tools, not a simple replacement of human judgment with systems that fail to meet basic requirements,” the study adds.