New Delhi: All relevant AI players in the world, irrespective of how well they get along with each other, must agree to slow down or completely pause further development of models, Anthropic CEO Dario Amodei said, warning against the potential cyber risks.
“The pace of AI progress is very fast. We’ve talked about the employment effects. I think society can adapt better if we slow things down a little bit. We see risks around cyber – as we’ve discussed with our Mythos model and Glasswing. We’re soon going to see risks around misuse of biology and around the models being autonomous and behaving in ways that humans don’t expect them to behave,” said Amodei.
On 4 June 2026, Anthropic posted a detailed report, When AI Builds Itself, which details how a growing share of AI development is now being carried out by other AI models. Anthropic argues that the trends point to a future wherein AI systems would be able to autonomously develop their own successors. This, the post explains, is called recursive self-improvement.
“AI that can build itself would be a major development in the history of technology—one that could bring enormous good for the world in science, healthcare, and beyond,” reads the post discussing the positive implications of furthering AI development. However, the post indicates the possibility of AI gaining even greater control than their human counterparts.
“But full recursive self-improvement also might increase the risks of humans losing control over AI systems. If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important.”
Also read: ‘AI will soon become so capable that I worry’: Anthropic CEO calls for urgent binding AI regulations
Geopolitical limitations
Anthropic argues that while recursive self-improvement is not an immediate reality, it could come sooner than one might expect. A pause, or a slowdown, might help institutions prepare for the upcoming challenges before they become the reality.
Amodei admits that geopolitical collaboration might be difficult, as all players compete to gain a superior position in the AI race, but they need to be willing to try.
“Now I think the US has solidified its lead in AI, and that puts us in a position where, if we on our side are more willing to put some guardrails in place. For example, make agreements about not developing biological weapons. Then it is possible that China may be willing to do the same,” Amodei noted in the interview. He notes that it would require mutual commitment and verifiability.