🔍 Read the full analysis: What The Gemini 4 Carbon AI Ranking Measures—and What It Leaves Out on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Gemini 4 Carbon scored 59 on Artificial Analysis’s Intelligence Index v5, one point ahead of Claude Opus 5.5. The result reflects performance across nine tests, not a universal measure of AI capability: Carbon leads four tests, trails on five, and is not yet generally available.
Gemini 4 Carbon leads Artificial Analysis’s new Intelligence Index v5 with 59 points, one point ahead of Claude Opus 5.5 and two ahead of GPT-6 Astra. The ranking measures results across nine benchmark tests at specified model settings; it does not establish that Carbon is best for every task, and Google has not yet made the model generally available.
Artificial Analysis launched the fifth version of its index on October 27, using four equally weighted categories: knowledge work, coding, science and general capability. Each category accounts for 25% of the total. The index combines nine tests, including business workflows, coding tasks, scientific work in a terminal and questions designed to test difficult reasoning and knowledge. The results supplied by the source put Claude Opus 5.5 at 58, GPT-6 Astra at 57, GPT-6.1 Sol at 55 and Google’s earlier Gemini 4 Argon at 50.
Carbon placed first on four of the nine tests, according to the source: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience and AutomationBench-AA. Its reported results include 69% on HLE-Diamond, compared with 68% for Claude Opus 5.5, and 65% on Terminal-Bench-Science, compared with 63% for GPT-6 Astra. It also scored 78% on AutomationBench-AA, slightly ahead of its predecessor.
The model did not lead the other five tests. The source reports stronger results by Claude models on office-style knowledge work and real-world coding, by GPT-6 Astra on a physics research benchmark, by Kimi K3.1 Pro on terminal tasks, and by Meta’s Muse Wonder Full on document reasoning. It also gives an estimated cost of $7.76 per index task at standard prices. That is below the stated $10.04 for Claude Opus 5.5 at maximum reasoning, but above GPT-6 Astra’s $6.24 and GPT-6.1 Sol’s $1.27. These figures depend on the benchmark’s workload and the listed model settings; they are not a universal cost for using each model.
Infographic 01 · What the #1 is made of
Number one overall. First in four of nine tests.
The Intelligence Index is a decathlon score: nine tests, one number. Carbon wins on reasoning, science and careful knowledge, and trails on office work and coding.
The nine tests behind the score
Carbon at its highest setting (high) against the best model on each test.
| Test | Weight | Carbon | Leader or runner-up |
|---|---|---|---|
| HLE-Diamond Hard reasoning + knowledge questions | Weight 10% | Carbon 69% | Carbon leads Next: Claude Opus 5.5, 68% |
| Terminal-Bench-Science Science research work in a terminal | Weight 15% | Carbon 65% | Carbon leads Next: GPT-6 Astra, 63% |
| AA-Omniscience v1.1 Knowledge, wrong answers penalized | Weight 15% | Carbon 54 | Carbon leads Next: Claude Opus 5.5, 48 |
| AutomationBench-AA Business workflows across apps | Weight 5% | Carbon 78% | Carbon leads Next: Gemini 4 Argon, 78% |
| AA-Briefcase Office-style knowledge work | Weight 15% | Carbon 1,609 Elo | Behind Claude Sonnet 5.5, 1,812 |
| FrontierCode 2.0 Code a maintainer would merge | Weight 15% | Carbon 41% | Behind Claude Opus 5.5, 46% |
| CritPt v2 (hard) Research-level physics | Weight 10% | Carbon 63% | Behind GPT-6 Astra, 71% |
| Terminal-Bench 4.0 Agentic terminal tasks | Weight 10% | Carbon 65% | Behind Kimi K3.1 Pro, 67% |
| GDP.pdf Reasoning over documents | Weight 5% | Carbon 28% | Behind Muse Wonder Full, 33% |
The surprise in the knowledge test
- Accuracy: Claude Fable 5.1 68%, Claude Opus 5.5 66%, GPT-6.1 Sol and GPT-6 Astra 62%, Carbon 58%
- Hallucination rate: Carbon 11%, Claude Opus 5.5 and GPT-6 Astra about 50%
- Carbon wins because it rarely bluffs, not because it knows the most
Read the weights
- Each of the four blocks counts 25%: knowledge work, coding, science, general
- Carbon trails in two of the four heaviest tests (15% each): office-style work and real-world code
- Hallucination rate = wrong answers among all questions not answered correctly
Source: Artificial Analysis, 28 Oct 2026; Index v5 announcement, 27 Oct 2026; Artificial Analysis model data, retrieved 29 Oct 2026. Scores rounded as in the article; Carbon tested at high, other models at their highest tested setting.
Powered by Thorsten Meyer AI
A Narrow Lead, Not a Universal Winner
The one-point gap makes the ranking a close result, not a clear separation in overall capability. For businesses comparing systems, the category results and the work they need done may matter more than the single combined score. A model that ranks lower overall could perform better on a company’s particular coding, document or office workflow.
Cost also changes the practical comparison. The source’s per-task estimates suggest Carbon costs less than Claude Opus 5.5 under the listed settings, but more than both GPT alternatives. Actual spending will depend on how much work a task requires, the model’s reasoning setting and the tools used alongside it. Organizations should compare cost per completed task on their own workloads rather than assume that the index estimate predicts their bill.
The result is less immediately relevant to ordinary users because Carbon is not publicly available yet. Google’s stated rollout begins with a subset of cyber defenders, followed by trusted partners and later developers, businesses and consumers. No dates for those stages are provided in the source. The ranking therefore describes a model that many readers cannot yet test for themselves.
Products Worth Considering
Apple MacBook Pro 16.2-inch with M5 Pro, 18-core CPU, 20-core GPU, 48GB Memory, 1TB SSD, Wi-Fi 7, Space Black
Apple MacBook Pro 16-inch (2021) – M1 Pro Chip, 10-Core CPU, 16-Core GPU, 16GB RAM, 1TB SSD, Space Gray
Apple MacBook Pro (14.2-inch, 2025) – M5 chip, 16GB Unified Memory, 1TB SSD, Space Black
As an affiliate, we earn on qualifying purchases.
How the Nine-Test Index Works
Artificial Analysis describes the index as a combined score across four equally weighted blocks. Knowledge work includes AA-Briefcase, AutomationBench-AA and GDP.pdf; coding includes FrontierCode 2.0 and Terminal-Bench 4.0; science includes Terminal-Bench-Science and the hard subset of CritPt v2; and general capability includes AA-Omniscience and HLE-Diamond. Individual tests carry different weights within their blocks, so a model’s score reflects both performance and the index’s chosen weighting.
The source says 60% of v5 uses private test sets or held-out answers, intended to reduce the chance that models succeed by memorizing publicly available questions. Many tasks also involve longer agent workflows, such as operating in a terminal, producing reports or fixing code. Those design choices make the index different from a simple quiz, while still leaving it a particular selection of tasks rather than a complete measure of everyday usefulness.
One result illustrates how scoring rules shape rank. Carbon reportedly scored 58% for plain accuracy on AA-Omniscience, behind several Claude and GPT models. It nonetheless led that test with 54 points because the scoring penalizes wrong answers. Artificial Analysis reports an 11% hallucination rate for Carbon on the test, compared with about 50% for Claude Opus 5.5 and GPT-6 Astra. The source also says Gemini 4 Argon’s rate was 4%, meaning Carbon was less cautious than its predecessor by this measure. These figures apply to this benchmark’s definition and results, not to every kind of question.
“The initial rollout is for “a subset of cyber defenders.””
— Google, as described in the supplied source
What the Score Cannot Settle
The index cannot establish which model is best across all uses. Its ranking depends on nine selected tests, their weights, and the model settings used. The supplied material does not provide the full test methodology, uncertainty ranges or enough information to assess how stable a one-point difference is across repeated runs. It also does not establish how the ranking would change under different settings or a different mix of tasks.
The cost comparison is similarly limited: the source gives per-index-task estimates but does not specify in the excerpt how those costs vary across task lengths, tool calls or usage patterns. Nor does it provide independent information about the reliability of the reported results beyond describing the benchmark design. The availability sequence is attributed to Google, but no public release date is stated. The source’s discussion of a separate Coding Agent Index is also incomplete, ending partway through a sentence, so no further claim from that passage can be confirmed here.
Broader Access and Real-Work Tests
Google’s stated next step is to provide Gemini 4 Carbon to a subset of cyber defenders, then to trusted partners, followed by developers, businesses and consumers. No timeline has been announced in the supplied material. Until access expands, most users cannot independently compare its performance with competing models in their own workflows.
For organizations, the useful follow-up is to test the model on representative tasks and track accuracy, completion rates, human review and cost. Future benchmark updates or fuller methodology could also clarify how much confidence to place in a one-point lead and how the score compares with other settings. For now, the confirmed result is a narrow first place on one defined index—not a finding that Carbon is the strongest or most economical choice for every user.
Key Questions
What does Gemini 4 Carbon’s 59-point score mean?
It is Carbon’s overall result on Artificial Analysis’s Intelligence Index v5, which combines nine benchmark tests across knowledge work, coding, science and general capability. It is not a universal score for all AI tasks.
How far ahead is Carbon?
Carbon scored 59 points, one point ahead of Claude Opus 5.5 at 58 and two ahead of GPT-6 Astra at 57, according to the supplied results.
Does Carbon perform best on every benchmark?
No. The source says Carbon leads four of the nine tests and trails on five, including tests for office-style knowledge work and real-world coding.
Can the public use Gemini 4 Carbon now?
Not yet, according to the supplied source. Google is starting with a subset of cyber defenders and plans later access for trusted partners, developers, businesses and consumers, but has not given dates.
Does the ranking prove Carbon is the best-value model?
No. The source estimates Carbon at $7.76 per index task under standard prices, but cost depends on the workload and model settings. The index does not establish which model will be cheapest for a particular organization.
Source: ThorstenMeyerAI.com
