Anthropic
Claude’s creator wants AI labs, including the company itself, to prepare for a coordinated slowdown if the models start building their successors. Critics are not convinced this will happen.
Companies on the front lines of artificial intelligence must be ready to slow downdefends one of the fastest growing companies in this sector.
A, creator of the chatbot Claudeclaims that AI systems may be on the verge of what he calls recursive self-improvementthe point at which they can Design and build your own successors with little human intervention.
The company says this evolution could increase the risk of humans losing control of the technology.
“We believe it would be positive for the world to have the option to slow down or temporarily suspend the development of AI, to allow social structures and research to align with the advancement of technology,” Anthropic says in a June 4 blog post titled “.”
The proposal highlights a hard problem in AI governance. A slowdown would require that andrival companies and governments from several countries to accept the same limits at the same timewithout a treaty that would oblige them to do so and in a context of increasingly intense competition.
This makes the warning technically important and politically delicate, notes the magazine: Anthropic is ask for a race to take place in which it continues to occupy a prominent position.
The speed at which technology is developing could have “huge implications” for society, and Anthropic points out your own operation as warning sign.
The company claims that Claude now writes more than 80% of the code of its systems, when before the launch of Claude Code, in early 2025, this percentage was few percentage pointsand that its engineers produce around 8 times more code per quarter than a few years ago.
At each stage of building AI, the human role is decreasing. “We haven’t reached that point yet, and recursive self-improvement is not inevitable“, says the company. “But it may come sooner than most institutions are prepared to face.”
Anthropic proposes what it calls “a global coordination mechanism” to slow down or even suspend AI development and give society room to adapt.
The company was sparse on details. He pointed to arms control agreements on intermediate-range nuclear missiles as a rough model.
For a suspension of this type to continue, the main companies in the sector would have to participate, and it would have to there is a credible way to demonstrate that they had, in fact, slowed down.
“I don’t think this is a genuine call for slowdown,” he says Noah Giansiracusaassociate professor of Mathematics at Bentley University and author of two books on algorithms and society. “We have already read the posts on Dario Amodei’s blog. he wants to continue moving forward at full speede”.
Anthropic did not respond to Scientific American’s questions about how this brake would work, nor to questions criticism that you are exaggerating what your systems can do.
Giansiracusa also considers that a break is impractical. “It’s literally impossible,” he says. “There is no chance of a slowdown. And I’m not even talking about China. Elon Musk would never slow down”.
The proposal fits into a standard that leaves some researchers standing back. Two months ago, Anthropic presented the Mythosa model he refused to make publicly available, claiming that It was too effective to find software vulnerabilities to be placed in anyone’s hands.
The call for a pause also comes just days after the company confidentially presented documentation for an initial public offeringand not long after a round of financing that valued the company at close to one billion dollars.
For skeptics, such eye-catching ads can be read as a business strategy: a way to attract regulatory scrutiny to the technological frontier as Anthropic keeps running towards her.
Mark Riedlprofessor in the School of Interactive Computing at the Georgia Institute of Technology, wrote in Bluesky that “the big AI companies are all catch the enthusiasm train around ‘recursive self-improvement’”.
Anthropic says it will spend the next few months bringing together governments, researchers and rival AI companies to understand whether a coordinated slowdown could work in practice.
“I don’t really see any reason to worry.” says Giansiracusa. “They’re flirting with the idea of — the idea that this changes everything — and I just don’t see it. I see technology continuing to progress.”
“Maybe things will speed up; maybe not Evidence cited by Anthropic (more AI-written code) suggest that the technology is usefuland not that it represents a big leap”, concludes Giansiracusa.
