Anthropic, the tech giant recently valued at $965 billion and headed for an IPO as early as this year, has issued a stark warning that humanity faces a looming risk of losing control over artificial intelligence (AI).
In a blog post published Thursday, the company revealed that its flagship AI model, Claude, now writes over 80% of its own code — a staggering surge from less than 10% in February last year.
According to Anthropic co-founder Jack Clark and Anthropic Institute lead Marina Favaro, the technology is rapidly accelerating toward “recursive self-improvement,” the critical threshold where an AI system can autonomously design, test, and develop its own superior successor without human intervention.
“When I look down at the car we’re driving, all I have is a gas pedal. I don’t have a brake pedal, and surely at some point in the future we might want that option,” Clark told CNN on Thursday night.
The company’s internal metrics illustrate the blistering pace of this evolution.
Beyond Claude’s overwhelming contribution to its own codebase, Anthropic engineers are currently absorbing eight times as much merged code daily compared to two years ago. A March 2026 internal survey revealed that employees produce roughly four times as much output with AI assistance than without it.
While Anthropic noted that self-improving AI could yield unprecedented breakthroughs in science and healthcare, it emphasized that the rapid transition creates severe societal risks. To manage these implications, the firm is calling for a coordinated, verifiable pause or slowdown in frontier AI development across the global tech sector.
However, the safety-focused lab stopped short of executing a unilateral halt. The authors argued that a single company pausing would simply shift the market lead to less cautious competitors without addressing systemic dangers. Instead, a meaningful moratorium would require multiple frontier labs across various countries to agree to identical conditions and establish airtight verification mechanisms.
The company acknowledged the immense geopolitical obstacles to such an agreement, comparing the challenge to historical nuclear diplomacy. “Training runs are far easier to conceal than missile silos… and the incentive to defect quietly is enormous,” the blog post said, noting that establishing international trust typically takes decades.
“Code authorship crossing 80% at a frontier lab pushes generation velocity past what human review can absorb. As verification shifts from people to agents, development approaches a threshold where no human checks code before it merges,” said Mitch Ashley, vice president and practice lead for Software Lifecycle Engineering and AI-Native Software Engineering at The Futurum Group. “While Anthropic’s blog post raises associated safety concerns, (it) doesn’t let Anthropic off the hook, whether that code is created by Anthropic or customers using their technology. The evidence and safety layers must exist before humans fully step back.”
The warning arrives during a highly consequential week for Anthropic. Following a massive $65 billion funding round in late May, the startup filed a confidential IPO prospectus with the Securities and Exchange Commission on Monday.
Despite its safety rhetoric, the company has faced intense pressure.
Earlier this year, Anthropic’s Mythos model disrupted the banking and software sectors by autonomously identifying code vulnerabilities. Furthermore, a recent executive order from the Trump administration shifted AI oversight to voluntary self-testing by labs, emphasizing national competitiveness. Under these pressures, Anthropic recently walked back a key safety pledge, admitting it would no longer withhold powerful models if global rivals were poised to match them.
In the coming months, the Anthropic Institute plans to convene policymakers, researchers, civil society groups, and rival tech firms to design risk-management frameworks and discuss the feasibility of international compliance infrastructure.
Competitors OpenAI, xAI, Alphabet, Meta Platforms Inc., and Mistral did not immediately respond to requests for comment.
