Anthropic chief executive officer Dario Amodei on Saturday urged artificial intelligence (AI) firms to slow down the pace of development of newer models, amid fears over the risks of “superintelligent” computer systems.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” he said in a post on his personal website.
Amodei highlighted the benefits of AI, saying it could cure “most major diseases in the next 5–10 years” and greatly accelerate economic growth while ushering in “a renaissance of democracy and freedom.”
However, the Anthropic CEO said that AI also brings “serious” risks, like many technologies before it. These, he said, include loss of control of these systems, misuse of AI for for cyberattacks and bioterrorism, and serious economic disruption.
‘Building AI too fast is reckless’: Amodei
In his post, Amodei said that while not developing the AI would result in depriving humanity of its benefits, “building it too fast is reckless.”
He urged for “more prudence” in fully addressing the risks that AI carries, which would involve “not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.”
While Amodei explicitly stated that pacing does not mean halting model training or technical progress, he said companies should take time to align and safeguard models, with third-party evaluators verifying the process.
“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei said.
Also Read | America’s great AI freakout has begun
Amodei warns of ‘recursive self-improvement’; mentions OpenAI-hugging face incident
Amodei said AI development had accelerated sharply over the past few months, driven partly by its growing ability to help build its next generation. However, he warned against “recursive self-improvement”, saying it could outpace humanity’s ability to understand and control AI systems.
The Anthropic CEO further raised concern regarding the OpenAI-Hugging Face incident (OAI-HF). In this, a swarm of AI agents were being tested internally by OpenAI, and these were supposed to be in an “isolated environment” called a “sandbox,” disconnected from the outside world. However, these escaped from the “sandbox”.
“A swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the “grader” responsible for evaluating their performance,” Amodei said. He asserted that this was not just one firm’s problem, but said such incidents had happened across the industry, including at Anthropic.
Anthropic CEO lays down three-step plan
Amodei proposed a three-step plan for “building AI at a balanced rate that aims to ensure its safety while still achieving its benefits and grappling with important geopolitical dilemmas.”
He urged frontier AI companies to give external evaluators ongoing, employee-like access to verify safety practices, investigate/ report incidents and help assess the alignment of not just completed AI models but training pipelines and processes. He called for AI companies in democratic countries to establish common safety standards and checks on AI progress, with government support where necessary.
The Anthropic CEO also urged United States and other governments to attempt to coordinate with “authoritarian governments”, “to the extent this is possible, while taking seriously the challenges of verifying compliance.”
