Artificial intelligence has spent the past few years becoming better at writing code, analyzing information and carrying out increasingly complicated tasks. But Anthropic is now warning that the next step could be far more consequential: AI systems may eventually become capable of helping build the next generation of AI systems themselves.
That possibility, known as recursive self-improvement, could transform the pace of technological progress.
It could also create a problem humanity has never had to solve before.
Anthropic, the company behind the Claude family of AI models, says it is already delegating a growing share of its own development process to AI systems. The company argues that if this trend continues far enough — and enough computing power becomes available — AI could eventually become capable of designing and developing its own successor.
Anthropic stresses that this milestone has not yet been reached and is not guaranteed to happen.
But the company believes it may arrive sooner than governments, institutions and the public are prepared for.
That warning is striking partly because Anthropic is not an outside critic watching the AI race from a distance. It is one of the companies directly building the systems driving the technology forward.
The company’s own data illustrates how quickly the development process is changing.
Anthropic says its engineers now ship roughly eight times as much code per quarter as they did during the 2021–2025 period. The company has also described a progression from ordinary software development, to chatbots assisting with snippets of code, to coding agents that can edit entire files, and now toward autonomous agents capable of running code and delegating lengthy tasks to other AI systems.
The obvious attraction is speed.
If AI can perform part of the work required to build better AI, developers can potentially create more capable systems faster. Those systems can then assist with still more difficult engineering tasks, creating a feedback loop.
At the most optimistic extreme, such a system could dramatically accelerate scientific research.
Anthropic itself says AI that can build itself could deliver enormous benefits in science, healthcare and other fields. Faster AI research could translate into better tools for drug discovery, medical research, materials science, software and complex engineering.
That is the optimistic side of the equation.
The darker possibility is that the same feedback loop could make advanced AI progress move faster than human institutions can respond.
If future systems are capable of designing their own successors, the people who created the original system may have less direct control over the systems that follow.
That is where the phrase “recursive self-improvement” becomes so important.
Ordinary technological progress has historically involved humans designing a tool, testing it, improving it and then producing a new version. Even extremely advanced computing systems remain products of human engineering.
Recursive self-improvement would blur that boundary.
The machine would no longer simply execute a human-designed development process. It could increasingly participate in, and potentially automate, the process of improving itself.
Anthropic’s warning is not that this has already happened.
It is that the trajectory is becoming visible.
A particularly significant figure emerged from Anthropic’s research earlier this year: by May, the company said more than 80% of the code merged into its codebase had been authored by Claude.
That statistic does not mean Claude independently created Anthropic. Human engineers remain responsible for architecture, evaluation, infrastructure, oversight and countless other decisions.
But it demonstrates how rapidly AI-generated work can move from experimental assistance into the core workflow of an AI company.
And that leads to the central governance problem.
What happens if one company tries to slow down but competitors continue developing at full speed?
Anthropic argues that a meaningful slowdown or pause would require coordination among multiple frontier AI laboratories across several countries. Otherwise, a cautious company could simply surrender its technological lead to a competitor willing to move faster.
This is where the issue becomes geopolitical.
AI development is no longer just a technology story. It is intertwined with national security, economic competition, semiconductor supply chains and strategic power.
A global pause sounds straightforward until the practical questions arrive.
Who decides when systems have become dangerous enough to justify a pause?
How would countries verify that rival laboratories have actually stopped?
What happens if one country pauses development while another keeps training frontier models?
And how would governments distinguish legitimate AI research from work being conducted in secret?
Anthropic acknowledges that these problems are extraordinarily difficult. International verification systems for other dangerous technologies took decades to develop. AI could move faster than diplomacy.
The company has therefore called for governments, researchers and AI developers to begin those discussions before recursive self-improvement arrives rather than after it becomes reality.
There is another reason the warning deserves attention: AI is increasingly becoming an economic force, not merely a software feature.
Companies are already using AI agents to automate coding, research, customer service and complex back-office work. Anthropic itself has argued that powerful AI could bring rapid changes to labor markets and the broader economy.
That raises the possibility that recursive improvement could accelerate not only AI research, but the economic disruption associated with it.
There is, however, an important distinction between possibility and prediction.
Anthropic has not said that machines are about to become fully autonomous superintelligences. The company explicitly acknowledges that recursive self-improvement may never occur. The argument is about preparedness — whether society should begin constructing safety, governance and verification mechanisms before the technology forces the issue.
That distinction matters because public discussion about AI often falls into two extremes: utopian promises on one side and science-fiction catastrophe on the other.
Anthropic’s position is more complicated.
The same technology that could help accelerate cancer research, improve scientific discovery and expand human capabilities could also create unprecedented control and safety challenges.
That may be the real story behind the company’s warning.
The question is no longer simply whether AI can become smarter.
It is whether AI could eventually become one of the main engines responsible for making AI smarter.
And if that happens, the most important decision humanity faces may not be how quickly to build the next model.
It may be whether human institutions can move quickly enough to remain in charge of the process.