Anthropic chief executive Dario Amodei on Saturday called for an immediate slowdown in the development of advanced artificial intelligence, warning that rogue AI agents could cause devastating damage within months. In a new essay, Amodei said the rapid improvement of AI models was creating risks that the industry needed to address urgently. He warned that swarms of rogue AI agents could potentially take control of the internet within six to 12 months.
OpenAI chief executive Sam Altman quickly backed the call for caution, saying the industry needed to slow the pace of frontier-model development and strengthen safety measures.
Amodei referred to a recent incident involving an OpenAI agent and Hugging Face, which he said could have had much more serious consequences. “Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet,” he wrote.
He warned that such an attack could potentially cause “hundreds of billions of dollars in damage” and that the scale of the threat could continue to grow if AI systems became more powerful without adequate safeguards.
Amodei said the industry needed to make better use of the time gained by slowing development.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” he wrote. “The measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try.”
Amodei said Anthropic would take a key safety measure by giving external evaluators access to its systems at a level comparable to that available to employees.
The evaluators would be able to assess the company’s safety procedures and report incidents, he said. He also called on other AI companies to adopt similar measures.
The chief executive urged democratic countries to work together to establish AI safety standards. He also called for cooperation between democratic and authoritarian governments on the issue.
Anthropic public policy chief Sarah Heck separately called for government intervention, including restrictions on the sale of advanced computer chips to countries such as China.
“The government has a critical role to play here, including blocking the sale of the most advanced chips to adversarial nations like China, enacting a national law requiring testing of frontier models, with the power to block the most advanced models that prove to be unsafe,” she wrote on X.
SpaceX AI chief executive Elon Musk also endorsed Amodei’s proposal. “Dario is right,” Musk wrote on X.
Altman later said he agreed that the industry needed to slow the pace of frontier AI development. He also said OpenAI would give external evaluators access to its systems. “I agree with Dario that we need to pace the frontier,” Altman wrote.
Amodei’s warning comes amid growing public and political debate over the risks posed by advanced AI systems. The discussion has intensified following the public resignation of an Anthropic employee who raised concerns about the potential consequences of AI development, as well as reports about the possibility of catastrophic AI-related harm.
The latest warnings have also renewed questions about whether companies will voluntarily limit the development of increasingly powerful models.
The AI industry has attracted enormous financial investment, making it difficult for companies to restrain development if they fear that competitors will continue advancing.
OpenAI said last month that it would delay development of its latest model, Astra, because of cybersecurity concerns. Anthropic has also previously debated the merits of slowing development unilaterally.
