Anthropic Warns Its Next Models May Hit Self-Improvement Threshold
3 min readAnthropic has told investors that its next generation of models may soon cross the line its own safety framework draws around self-improving AI. In an amended S-1 filed with the SEC on November 2, the company added new risk factor language stating that internal estimates for future models, not current ones, indicate it “may imminently meet” the Responsible Scaling Policy definition of dramatically accelerated progress attributable to AI. The Anthropic self-improving AI warning is one of the most unusual disclosures ever to appear in an IPO filing.
Background: What the Threshold Means
Anthropic’s Responsible Scaling Policy sets capability thresholds that trigger stronger safeguards. One of them covers automated AI research. According to The Neuron, the threshold is met if AI can replace the company’s entire research staff at a competitive cost, or if overall progress roughly doubles relative to both the expected pace and the fastest sustained historical pace, with automated research substantially responsible for the jump.
What Anthropic Disclosed
As reported by TechCrunch, the filing says Anthropic has not designated any current model as exceeding the threshold and is still assessing the data. It also admits the company has in the past failed to identify a significant acceleration until after it occurred.
The filing warns that using AI to speed up Anthropic’s own research may push model capabilities ahead of its ability to build and validate matching safety, alignment and security measures. Anthropic states that it does not expect to safely let Claude fully autonomously design and build successive frontier models without meaningful human supervision, and that it is considering updates to the Responsible Scaling Policy to keep humans in control of frontier development.
The same filing reports that Claude was responsible for more than 90 percent of the company’s code contributions by September, with daily code output per engineer in the third quarter running at more than 45 times its 2024 level. CNBC reported that the amended S-1 shows third quarter revenue of roughly $17 billion alongside a net loss of about $23 billion.
Why It Matters
Recursive self-improvement has long been treated as a theoretical milestone. A frontier lab putting it in a securities filing as a near-term risk changes the conversation. Investors in the ANT listing must weigh a company that says its product may soon accelerate its own development faster than its safety work can follow.
Skeptics note that an S-1 is also a marketing document, and independent evaluator METR judged Claude Opus 5.5 unlikely to automate the entire research process, estimating acceleration at about 1.5 times. Watch for the promised Responsible Scaling Policy update and for how regulators respond before the IPO prices.
Anthropic is asking the public markets to fund a race it says it may not be able to fully steer. That tension will define the ANT listing.