🔍 Read the full analysis: METR On Anthropic: AI Development Findings And Missing Details on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
METR’s November 5 assessment estimates that Anthropic’s progress on well-specified AI tasks accelerated to about 1.9 times its earlier pace, with AI assistance likely responsible for much of the change. The nonprofit says Anthropic redacted a majority of the evidence it found most informative, limiting public scrutiny of the findings.
AI-evaluation nonprofit METR estimated that Anthropic’s model-development progress accelerated about 1.9 times after October 2025, based on an assessment using internal company data. The report, published November 5, says AI assistance likely accounts for much of the acceleration, but also says Anthropic redacted a majority of the evidence METR considered most informative, leaving important parts of the analysis difficult for outsiders to verify.
METR’s central estimate compares Anthropic’s rate of improvement on a combined measure of model capabilities before and after an estimated break point of October 7, 2025. The group calculates that the measure rose by about 15 points per year before the break and about 29 points per year afterward. METR describes the result as roughly two years of earlier progress occurring in one year. Its estimate is 1.9 times faster, with a reported 95% confidence interval of 1.5 to 2.8 times.
The estimate is not a direct measure of how quickly researchers work, nor does it establish that AI systems are independently creating their successors. METR says it is assessing progress on well-specified tasks. Anthropic’s policy defines a related warning threshold as progress equivalent to two years at the earlier pace in one year, where the acceleration is plausibly attributable to automated research or engineering. METR says it would not be surprised by a result from 1.3 to 3 times the earlier rate, reflecting uncertainty in the estimate.
METR’s assessment drew on benchmark results for unreleased models, training metrics, surveys and about 15 staff interviews. The nonprofit said roughly 2.5 full-time-equivalent staff worked on the assessment over 11 weeks, using Anthropic devices. METR said it received no payment; Anthropic supplied laptops and free tokens and retained the right to redact non-public information. METR estimates that perhaps about 80% of the acceleration may be due to AI use, while saying it is confident the change did not result from faster growth in training compute.
Infographic 01 · How fast is AI speeding up AI?
About twice the old pace, most likely because of AI
An outside evaluator's estimate from internal Anthropic data, plus Anthropic's own measurements. Wide uncertainty, much of the evidence redacted.
Acceleration of Anthropic’s benchmark progress (AECI) since October 2025, 95% confidence interval 1.5x to 2.8x. METR’s overall judgment: around twice the old rate; it “would not be surprised” by 1.3x or 3x.
Capability points per year
AECI trend of Anthropic models, break point 7 Oct 2025
All labs before October 2025: +17 a year, so 1.4x. Different task-difficulty assumptions: 1.1x to 2.9x.
Sources: METR assessment, 5 Nov 2026 · METR risk report review · Anthropic Institute measurements.
Powered by Thorsten Meyer AI
The Implications of Faster AI Research
The report addresses a question relevant to AI safety and governance: whether AI tools are already helping developers make more capable systems faster, potentially shortening the time available to study risks and respond. If the estimate is broadly right, AI is becoming a larger part of the development process, though the findings do not show that AI systems are autonomously setting research goals or building successor models without human direction.
METR also reports that Anthropic’s own measurement found Claude led 26% of its AI research and development work in August 2026, compared with less than 1% in February. The source material says about 30,000 agents were working at a time on Anthropic’s main internal platform. These figures concern Anthropic’s internal measures and should not be read as a claim that AI performs 26% of all tasks, or that every agent operates independently.
The public evidence matters because Anthropic’s Responsible Scaling Policy includes an early-warning threshold for sharply accelerated automated research and engineering. METR considers it plausible that the threshold may have been crossed by July 15, 2026. That is an assessment, not confirmation that Anthropic formally determined the trigger was met or that a particular safeguard was activated.
Products Worth Considering
Apple MacBook Pro 16.2-inch with M5 Pro, 18-Core CPU, 20-Core GPU, 48GB Memory, 1TB SSD, Nano-Texture Display, Space Black
Apple MacBook Pro 14-inch (2021) – M1 Pro Chip, 10-Core CPU, 16-Core GPU, 16GB RAM, 1TB SSD, Space Gray
Apple MacBook Pro 14.2-inch with M5 Max, 18-Core CPU, 20-Core GPU, 48GB Memory, 2TB SSD, Space Black
As an affiliate, we earn on qualifying purchases.
How METR Measured Anthropic’s Progress
METR’s work differs from an evaluation of a publicly released model. Rather than testing only finished systems, the nonprofit examined internal indicators at Anthropic, including scores from unreleased models and information about training and research workflows. Its main capability measure, AECI, combines results across benchmarks into a single index. The report’s growth-rate comparison uses the change in that index over time.
METR’s 1.9-times estimate is higher than the 1.7-times figure in Anthropic’s Haiku 5.5 system card. METR says the difference mainly reflects its inclusion of newer models that Anthropic had not released. Looking across frontier models from multiple developers, rather than only Anthropic’s, METR estimates a smaller increase: from about 17 to 24 index points per year, or 1.4 times. It offers two possible explanations for the gap: Anthropic may have been behind the wider frontier in 2025, or its earlier models may have been less tuned to benchmarks.
Anthropic’s Responsible Scaling Policy version 3.4 took effect July 8, 2026. It describes automated research and development triggers, including a doubling in progress plausibly linked to research or engineering automation. Anthropic’s risk report characterizes that threshold as a potential early warning. The policy and the METR estimate are related, but they are not the same thing: METR’s analysis does not by itself establish a policy determination.
““perhaps ≈80%””
— METR
What the Redactions Leave Unverified
The largest limitation is that Anthropic redacted a majority of the evidence METR found most informative. The source material does not identify all withheld data or show how each redaction affects the ability to reproduce the analysis. METR also dropped a planned examination of how Anthropic divides compute between safety work and capability development, leaving that allocation unanswered.
The 1.9-times result depends on choices about benchmarks, model selection and task difficulty. METR says alternative assumptions about task difficulty move the estimate from 1.1 to 2.9 times; most other checks remain near two times. The report also presents competing interpretations of the difference between Anthropic-only and industry-wide rates. The material provided does not establish which interpretation is correct.
It is also unclear from the source whether Anthropic formally concluded that its policy threshold had been crossed. An SEC filing dated November 2 said Anthropic “may imminently meet” its definition of dramatically accelerated progress. That language signals a possibility, not a final finding. METR’s view that the threshold may have been reached by July 15 is its assessment, and further details about any resulting policy actions are not provided.
What Could Clarify the Findings
The next useful development would be greater public access to the evidence behind METR’s analysis, or a clearer account from METR and Anthropic of how the redactions affect confidence in the findings. The source material does not give a date for further disclosures or identify a scheduled follow-up assessment.
Readers can also watch for an explicit update from Anthropic on whether it has determined that a Responsible Scaling Policy trigger was met and what measures, if any, followed. Until such information is available, the report supports a careful conclusion: METR found evidence consistent with faster AI-development progress at Anthropic, but the extent of AI’s causal contribution and the policy consequences remain partly unverified. The estimates describe a specific company and assessment period, not the pace of progress across the entire AI industry.
Key Questions
What did METR estimate about Anthropic’s AI progress?
METR estimated that Anthropic’s progress on a combined capability measure accelerated about 1.9 times after an estimated October 2025 break point. Its reported 95% confidence interval is 1.5 to 2.8 times.
Does the report say AI is building new AI without people?
No. The findings concern AI assistance in research and development. The source describes humans as still choosing goals and distinguishes current AI-assisted work from fully autonomous development of a successor system.
How much of the acceleration did METR attribute to AI?
METR said perhaps about 80% may be due to AI. This is a qualified estimate, not a definitive measurement of causation.
Why are the findings difficult to check?
METR said Anthropic redacted a majority of the evidence it found most informative. The source also says METR dropped a planned analysis of how Anthropic allocates compute between safety and capability work.
Did Anthropic confirm that its policy threshold was crossed?
The material provided does not establish that. METR said crossing the threshold by July 15 was plausible, while an Anthropic filing said the company “may imminently meet” its definition. Neither statement is a reported final determination.
Source: ThorstenMeyerAI.com
