METR says Anthropic's model progress has roughly doubled in speed

METR, a Berkeley nonprofit that tests AI models, has published an outside assessment saying that Anthropic is improving its models about twice as fast as it was a year ago. Anthropic invited the review while it prepares what may become the largest stock-market flotation ever, due later this month. METR released the findings on 5 November, together with a second document that criticises the public version of Anthropic's August risk report for understating the speed-up.

The review ran for about eleven weeks, from 12 August to 29 October, with roughly two and a half full-time METR staff and some fifteen interviews with Anthropic researchers. It also covered three unreleased models that finished training between July and October. On METR's composite benchmark score, Anthropic gained about 15 points a year between March 2024 and October 2025 and about 29 points a year since, a factor of roughly 1.9. Put differently, the past year brought as much benchmark progress as the two years before it, about the distance from Claude 2.1 to Claude Sonnet 4.5.

“the public has to rely on METR’s judgment much more than would be ideal”

METR, in the redaction summary of its report, in its report · metr.org ↗

METR's best guess is that around 80% of the current speed-up comes from AI being used inside Anthropic's own research, and it is confident that growth in computing power is not the explanation. It gives a plausible range of 1.3 to 3 for the acceleration, and when OpenAI, Google and xAI models are counted the industry-wide figure drops to about 1.4. Anthropic's public August report said the acceleration was below a factor of two, while the unredacted text implies 1.97. METR says the warning threshold in Anthropic's own Responsible Scaling Policy, a doubling from automated research, may have been crossed as early as 15 July.

The report is cautious about its own evidence. A poll of Anthropic staff in October found that people felt about twelve times more productive with the latest internal model, a figure METR calls loose and likely to overstate real gains. Inside the company views differ: one researcher puts AI's share of the progress at only 15 to 20%, and another credits new hires. METR also says the result could overstate things if it holds only for well-defined tasks that can be graded automatically.

The larger complaint is about access. Anthropic held unilateral redaction rights and removed most of the evidence METR rated as most informative, including forecasts for future models, training-step breakdowns and a bottleneck analysis. One chart in the report is stamped as fake data because the real figures were withheld. Anthropic says the cuts remove confidential details that could speed up rival developers. METR also planned to examine how Anthropic divides compute between capability and safety work, but Anthropic cancelled that part midway and may return to it later.

The findings land after Dario Amodei's 12 September essay urging the industry to slow down, which named METR as a good embedded outside evaluator, and after Sam Altman said he backed independent evaluators with employee-like access. METR shares Anthropic's view that the immediate risk of catastrophe is low, but it found no bottleneck that would stop the pace continuing for twelve months and cannot rule out another sharp jump within six. Daniel Kokotajlo, author of the AI 2027 scenario, argued that auditing how compute is allocated is what would make any slowdown pledge verifiable.