New Delhi: Anthropic researchers have warned that AI models could soon be able to replicate themselves autonomously, even without access to external data centres.
New research from Anthropic’s Frontier Red Team has found that small artificial intelligence models could independently copy themselves across commercial Internet of Things (IoT) devices, such as smart TVs, fridges, vacuum cleaners and ovens.
The findings, published on 12 October and authored by researchers William Zhou and Daniel Freeman, report that models with around 9 billion parameters already show a strong ability to set up and run inference (the process of generating outputs) on a wide range of unfamiliar hardware. By the end of this year, smaller models, too, could achieve full end-to-end self-replication.
“Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial internet-of-things (IoT) devices, like smart ovens and off-brand Android TV boxes,” the research paper states.
The research also highlights the risks posed by such replication.
“A replicator like this would be challenging to shut down, since there’s no centralized point to shut off—though it may be slow (with a doubling time of days),” the researchers explain. “Unlike non-AI malware, it could also figure out unfamiliar devices on its own.”
Traditional computer worms copy code from one machine to another. An “AI worm” could carry its own model weights (or re-download them) and then run inference on newly infected hardware. This could result in autonomous replication and adaptation (ARA)—an AI system that deploys and spreads itself without human control.
The research focused on low-powered, poorly secured consumer devices.
In the experiments, smaller models succeeded in setting themselves up and running on a range of different devices. More capable models completed the full cycle more effectively, scanning a network, transferring themselves and launching new copies. Only the most advanced model among those tested achieved consistent end-to-end replication in a simplified environment.
“Only GLM-5.2 out of our set of tested open-weight models is capable of full end-to-end replication,” the paper states.
The researchers estimate that smaller models could reach a similar level of competence by the end of the year. By then, models small enough to run on ordinary IoT hardware could become capable of spreading on their own.
However, the spread is likely to be slow.
“As an analogy, imagine the replicator as a slime mold or a swarm of fruit flies, which replicate over days,” the paper states.
While a single replication cycle might take a day or two, a large number of infected devices could support many simultaneous copies of the AI. The resulting network could be comparable in scale to past IoT botnets that compromised millions of devices.
Unlike conventional malware, self-replicating AI malware might be able to figure out unfamiliar devices on its own. Traditional botnets usually target specific types of devices using fixed instructions. In the experiments, success rates improved when later versions of a model inherited notes from earlier attempts.
Anthropic also outlines several possible defences.
Unusual network traffic caused by model transfers and device-to-device communication could be detected by internet service providers. Quarantining infected networks, as was done to contain earlier botnets, has proved effective in the past. Stronger security defaults from device manufacturers, already required in some countries, could also reduce the pool of vulnerable targets.
“The capabilities that make AI models generally powerful coding agents also generalize to cyber tasks, including autonomous replication,” the researchers conclude. “Like coding agents in 2024, we’re seeing signs of life for AI worms. We don’t yet have a full rogue replicator, but agents are becoming increasingly capable of completing the individual steps in the replication chain.”
(Edited by Prashant Dixit)