What counts as plagiarism? AI-generated papers pose new dangers

This January, Byeongjun Park, a researcher in synthetic intelligence (AI), obtained a stunning e-mail. Two researchers from India informed him that an AI-generated manuscript had used strategies from considered one of his papers, with out credit score.

Park appeared up the manuscript. It wasn’t formally revealed, however had been posted on-line (see go.nature.com/45pdgqb) as considered one of plenty of papers generated by a instrument known as The AI Scientist — introduced in 2024 by researchers at Sakana AI, an organization in Tokyo¹.

The AI Scientist is an instance of absolutely automated analysis in pc science. The instrument makes use of a big language mannequin (LLM) to generate concepts, writes and runs the code by itself, after which writes up the outcomes as a analysis paper — clearly marked as AI-generated. It’s the beginning of an effort to have AI programs make their very own analysis discoveries, says the staff behind it.

The AI-generated work wasn’t copying his paper immediately, Park noticed. It proposed a brand new structure for diffusion fashions, the types of mannequin behind image-generating instruments. Park’s paper handled enhancing how these fashions are educated². But to his eyes, the 2 did share related strategies. “I was surprised by how closely the core methodology resembled that of my paper,” says Park, who works on the Korea Advanced Institute of Science and Technology (KAIST) in Daejeon, South Korea.

Researchers constructed an ‘AI Scientist’ — what can it do?

The researchers who e-mailed Park, Tarun Gupta and Danish Pruthi, are pc scientists on the Indian Institute of Science in Bengaluru. They say that the problem is larger than simply his paper.

In February, Gupta and Pruthi reported³ that they’d discovered a number of examples of AI-generated manuscripts that, in line with exterior consultants they consulted, used others’ concepts with out attribution, though with out immediately copying phrases and sentences.

Gupta and Pruthi say that this quantities to the software program instruments plagiarizing different concepts — albeit with no in poor health intention on the a part of their creators. “A significant portion of LLM-generated research ideas appear novel on the surface but are actually skillfully plagiarized in ways that make their originality difficult to verify,” they write.

In July, their work gained an ‘outstanding paper’ award on the Association for Computational Linguistics convention in Vienna.

But a few of their findings are disputed. The staff behind The AI Scientist informed Nature that it strongly disagrees with Gupta and Pruthi’s findings, and doesn’t settle for that any plagiarism occurred in The AI Scientist case research that the paper examines. In Park’s particular case, one impartial specialist informed Nature that he thought the AI manuscript’s strategies didn’t overlap sufficient with Park’s paper to be termed plagiarism. Park himself additionally demurred at utilizing ‘plagiarism’ to explain what he noticed as a robust methodological overlap.

Beyond the precise debate about The AI Scientist lies a broader concern. So many papers are revealed annually — particularly in pc science — that researchers already battle to maintain observe of whether or not their concepts are actually revolutionary, says Joeran Beel, a specialist in machine-learning and knowledge science on the University of Siegen, Germany.

And if extra LLM-based instruments are used to generate concepts, this might deepen the erosion of mental credit score in science. Because LLMs work partially by remixing and interpolating the textual content they’re educated on, it might be pure for them to borrow from earlier work, says Parshin Shojaee, a pc scientist on the Virginia Tech Research Center — Arlington.

The subject of ‘idea plagiarism’, though little mentioned, is already an issue with human-authored papers, says Debora Weber-Wulff, a plagiarism researcher on the University of Applied Sciences, Berlin, and she or he expects that it’ll worsen with work created by AI. But, in contrast to the extra acquainted types of plagiarism — involving copied or subtly rewritten sentences — it’s arduous to show the reuse of concepts, she says.

That makes it troublesome to see methods to automate the duty of checking for true novelty or originality, to match the tempo at which AIs are going to have the ability to synthesize manuscripts.

“There’s no one way to prove idea plagiarism,” Weber-Wulff says.

Overlapping strategies

Bad actors can, in fact, already use AI to intentionally plagiarize others or rewrite others’ work to cross it off as their very own (see Nature 2025). But Gupta and Pruthi puzzled if well-intentioned AI approaches may be utilizing others’ strategies or concepts too.

Gupta and Pruthi have been first alerted to the problem once they learn a 2024 research led by Chenglei Si, a pc scientist at Stanford University in California⁴. Si’s staff requested each folks and LLMs to generate “novel research ideas” on subjects in pc science. Although Si’s protocol included a novelty verify and requested human reviewers to evaluate the concepts, Gupta and Pruthi argue that a few of the AI-generated concepts produced by the protocol however lifted from present works — and so weren’t ‘novel’ in any respect.

They picked out one of many AI-generated concepts in Si’s paper, which they are saying borrowed from a paper first posted as a preprint⁵ in 2023. Si tells Nature that he agrees that the ‘high-level’ concept was just like materials within the preprint, however that “whether the low-level implementation differences count as novelty is probably a subjective judgement”. Shubhendu Trivedi, a machine-learning researcher who co-authored that 2023 preprint, and was till lately on the Massachusetts Institute of Technology in Cambridge, says that “the LLM-generated paper was basically very similar to our paper, despite some superficial-level differences”.

AI is complicating plagiarism. How ought to scientists reply?

Gupta and Pruthi additional examined their concern by taking the 4 AI-generated analysis proposals publicly launched by Si’s staff and the ten AI manuscripts launched by Sakana AI, and generated 36 recent proposals themselves, utilizing Si’s methodology. They then requested 13 specialists to attempt to discover overlaps in strategies between the AI-made works and present papers, utilizing a 5-point scale, on which 5 corresponded to a ‘one-to-one mapping in methods’ and 4 to ‘mix-and-match from two-to-three prior works’; 3 and a pair of represented more-modest overlaps and 1 indicated no overlap. “It’s essentially about copying of the idea or crux of the paper,” says Gupta.

The researchers additionally requested the authors of authentic papers recognized by the specialists to present their very own views on the overlaps.

Including this step, Gupta and Pruthi report that 12 papers of their pattern of AI-generated works reached ranges 4 and 5, implying, they stated, a plagiarism proportion of 24%; the determine rises to 18 (36%) if circumstances through which the unique authors didn’t reply are included. Some have been from Sakana’s and Si’s work, though Gupta and Pruthi talk about intimately solely the examples reported on this story.

They additionally stated they’d discovered an identical sort of overlap in an AI-generated manuscript (see go.nature.com/4oym4ru) that, Sakana introduced this March, had handed via a stage of peer overview for a workshop at a prestigious machine-learning convention, the International Conference on Learning Representations.

At the time, the agency stated that this was the primary fully-AI-generated paper to cross human peer overview. It additionally defined that it had agreed with workshop organizers to trial placing AI-generated papers into peer overview and to withdraw them in the event that they have been accepted, as a result of the group hadn’t but determined whether or not AI-generated papers must be revealed in convention proceedings. (The workshop organizers declined Nature’s request for remark.)

Gupta and Pruthi say that this paper borrowed its core contribution from a 2015 work⁶, with out citing it. Their report quotes the authors of that paper, pc scientists David Krueger and Roland Memisevic, as saying that the Sakana work is “definitively not novel”, and figuring out a second uncited manuscript⁷ that the paper borrowed from.

Another pc scientist, Radu Ionescu on the University of Bucharest, informed Nature he rated the similarity between the AI-generated work and Krueger and Memisevic’s paper as a 5.

Krueger, who’s on the University of Montreal in Canada, informed Nature that the associated works ought to have been cited, however that he “wouldn’t be surprised to see human researchers reinvent this and miss previous work” too. “I think this AI system and others are not capable of achieving academic standards for referencing related work,” he stated, including that the AI paper was “extremely low quality overall”. But he wasn’t certain whether or not the phrase plagiarism must be utilized, as a result of he feels that time period implies that the particular person (or AI instrument) reusing strategies was conscious of earlier work, however selected to not cite it.

Pushback

The staff behind The AI Scientist, which incorporates researchers on the University of Oxford, UK, and the University of British Columbia in Vancouver, Canada, pushed again strongly in opposition to Gupta and Pruthi’s work when requested by Nature. “The plagiarism claims are false,” the staff wrote in an e-mailed point-by-point critique, including that they have been “unfounded, inaccurate, extreme, and should be ignored”.

On two AI Scientist manuscripts mentioned in Gupta and Pruthi’s paper, for example, the staff says that these works have completely different hypotheses from these within the earlier papers and apply them to completely different domains, even when some parts of the strategies are associated.

Is it OK for AI to write down science papers? Nature survey reveals researchers are break up

The references discovered by the specialists for Gupta and Pruthi’s evaluation are work that the AI-generated papers may have cited, however nothing extra, the AI Scientist staff says, including: “What they should have reported is some related work that went uncited (a daily occurrence by human authors).” The staff says it might be “appropriate” to have cited Park’s paper. In the case of Krueger’s paper and the second uncited manuscript, the AI Scientist staff says, “these two papers are related, so, while it is an everyday occurrence by humans not to include works like this, it would have been good for The AI Scientist to cite them”.

Ben Hoover, a machine-learning researcher on the Georgia Institute of Technology in Atlanta who focuses on diffusion fashions, informed Nature that he’d rating the overlap with Park’s paper as a ‘3’ on Gupta’s scale. He stated the AI-generated paper is of a lot decrease high quality and fewer thorough than Park’s work, and will have cited it, however “I would not go so far as to say plagiarism.” Gupta and Pruthi’s evaluation depends on ‘superficial similarities’ between generic statements within the AI-generated work that, when learn intimately, don’t meaningfully map to Park’s paper, he provides. Ionescu informed Nature he would give the AI-generated paper a ranking of two or 3.

Park judges the overlap along with his paper to be a lot stronger than Hoover’s and Ionescu’s scores. He says he would give it a rating of 5 on Gupta’s scale, and provides that it “reflects a strong methodological resemblance that I consider noteworthy.” Even so, this doesn’t essentially align with what he sees because the authorized or moral definition of plagiarism, he informed Nature.

What counts as plagiarism

Part of the disagreement may stem from completely different operational understandings of what ‘plagiarism’ means, particularly on the subject of overlap in concepts or strategies. Researchers who research plagiarism maintain completely different views on the time period from these of a few of the pc scientists within the present debate, says Weber-Wulff.

“Plagiarism is a word we should and do reserve for extreme cases of intentional fraudulent cheating,” the AI Scientist staff wrote, including that Gupta and Pruthi “are wildly out of line with established conventions regarding what counts as plagiarism in academia”. But Weber-Wulff disagrees: she says that intent shouldn’t be an element. “The machine has no intent,” she says. “We don’t have a good mechanism for explaining why the system is saying something and where it got it from, because these systems are not built to give references.”

Weber-Wulff’s personal favoured definition of plagiarism is that it happens when a manuscript “uses words, ideas, or work products attributable to another identifiable person or source without properly attributing the work to the source from which it was obtained in a situation in which there is a legitimate expectation of original authorship”. That definition was produced by Teddi Fishman, the previous director of a US non-profit consortium of universities known as the International Center for Academic Integrity.