A new edition changes the evaluations and the rankings
Artificial Analysis published version 5 of its Intelligence Index on October 27, 2026. The composite index compares models across tasks intended to reflect current uses. This update changes both the evaluations and the context for interpreting the results: the organization says that 60% of the index is now based on evaluations using private test sets or held-out answers.
Artificial Analysis places Claude Opus 5.5, at maximum effort, at 58 points, ahead of GPT-6 Astra at 57 and GPT-6.1 Sol at 55. These are results published by the organization that designed and ran the index; the available evidence does not include an independent reproduction of the scores. A composite ranking also reflects a particular design: it does not establish that the top-ranked model is best for every task, price point or operating environment.
Two new evaluations and three updates
Version 5 adds FrontierCode 2.0, Cognition’s private benchmark of 132 tasks in open-source code repositories. It evaluates whether a repository maintainer would accept an agent’s change. The announcement says the tasks remain private to reduce contamination and that Artificial Analysis runs each task three times in its evaluation environment. That describes the stated procedure, but it does not establish that contamination has been eliminated or that third parties can reproduce the results.
The other new evaluation, Terminal-Bench-Science, contains 70 scientific research tasks carried out in a terminal environment. The announcement says experts curate and review the tasks, and that a task counts as passed only if all its tests pass. The update also adds HLE-Diamond, a 1,000-question selection split evenly between reasoning and knowledge; AA-Omniscience v1.1, with revisions to its question set; and a private, 40-question hard subset of CritPt v2.
What each announced change evaluates
These descriptions and weights summarize the design reported by Artificial Analysis; they are not an independent audit of the evaluations.
| Evaluation | Change or focus | Reported weight |
|---|---|---|
| FrontierCode 2.0 | Coding tasks in repositories; judged by whether the agent’s change would be accepted. | 15% |
| Terminal-Bench-Science | Scientific research workflows in a terminal environment. | 15% |
| HLE-Diamond | 1,000 questions: half reasoning, half knowledge. | 10% |
| AA-Omniscience v1.1 | Revised knowledge and hallucination benchmark. | 15% |
| CritPt v2 hard subset | 40 private research-level physics questions. | 10% |
How the index is weighted
Intelligence Index v5 assigns 25% of the total to each of four areas: knowledge work, coding, science and general evaluation. Within those areas, the weight is divided among nine evaluations. The overall result therefore depends both on performance in each test and on decisions about which tests to include and how heavily to weight them.
The composition matters when comparing editions. Artificial Analysis retains AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0; adds the new evaluations and updated versions; and removes GDPval-AA v2.1, SciCode and AA-LCR v1.1. A difference between v4.3 and v5 should not automatically be read as an equivalent gain or loss in general capability: the evaluation set changed, and models may also behave differently on the specific tasks included.
Weights by category
According to the methodology described in the announcement, each category accounts for 25% of the index.
| Category | Evaluations and weights |
|---|---|
| Knowledge work | AA-Briefcase: 15%; AutomationBench-AA: 5%; GDP.pdf: 5% |
| Coding | FrontierCode 2.0: 15%; Terminal-Bench 4.0: 10% |
| Science | Terminal-Bench-Science: 15%; CritPt v2 hard subset: 10% |
| General | AA-Omniscience v1.1: 15% (10% accuracy and 5% non-hallucination); HLE-Diamond: 10% |
The gap between open-weights models widens in this measurement
In the published table, Kimi K3.1 Pro scores 46 points and ranks as the top open-weights model, although the announcement notes that its weights had not yet been released. Artificial Analysis compares that score with Claude Opus 5.5’s 58 and reports a 12-point gap, compared with 3 points in v4.3. This comparison reflects the scores and editions presented by the index’s publisher; by itself, it does not show that the index changes caused the gap to widen.
Among models whose weights the announcement identifies as released, DeepSeek V4.1 Flash 1017 and GLM-5.3 lead that group at 32 points each. In this particular ranking, the label “open weights” does not necessarily mean that the weights are already available: Kimi illustrates why it is useful to distinguish the category used for comparison from a model’s actual availability.
An index cannot replace evaluation for a specific use case
The index also shows that no single model leads every evaluation. In the published results, Claude Opus 5.5 leads FrontierCode 2.0, HLE-Diamond and AA-Omniscience v1.1; GPT-6 Astra leads Terminal-Bench-Science and the CritPt v2 hard subset. Other models rank first in specific tests of knowledge work, automation, document generation or terminal tasks. This spread shows how an overall ranking can obscure strengths and weaknesses on particular tasks.
Cost and configuration also affect practical comparisons. Artificial Analysis reports substantial differences in cost per task among leading models and says that effort settings can be associated with marked changes in scores and cost. These figures depend on the provider’s stated execution conditions and do not guarantee the same performance or price in a particular application. When choosing a model, compare representative tasks, quality requirements, latency, cost and deployment conditions relevant to your own use.
How to use the ranking in a purchasing decision
The index is a comparative reference, not a guarantee of results in a particular application.
- 01Define your real tasks and success criteria before choosing a model.
- 02Identify which evaluations resemble those tasks and how much they contribute to the overall score.
- 03Compare models under the same effort setting and available conditions, and assess cost, latency and deployment constraints separately.
- 04Test a representative sample of your own cases; a high index position does not establish performance on data or workflows that were not evaluated.
Transparency and limits of the published evidence
Private test sets can reduce public exposure of questions and answers, but they also limit the ability of third parties to repeat the evaluations exactly. The announcement identifies private evaluations and describes some safeguards, such as keeping FrontierCode 2.0 tasks private and removing public or saturated questions from AA-Omniscience v1.1. The available evidence does not describe contamination controls across all evaluations in sufficient detail to confirm their effects, so it would not be justified to say that the tasks are immune to contamination.
A cautious reading is to treat v5 as a composite measurement published by its owner, useful for comparing models under that protocol and its weighting choices. Specific scores, comparisons with earlier editions and gaps between model categories should be understood within that scope. Publishing the index is not the same as an independent replication, and the ranking cannot automatically be generalized to every task, user or configuration.
Open questions
- The scores and rankings cited come from Artificial Analysis’s announcement; the available evidence contains no independent replication of the results.
- The announcement describes measures related to task privacy and dataset revision, but the available evidence does not independently establish how contamination is controlled across all evaluations.
- The difference between v4.3 and v5 does not isolate the effects of each methodological change, weighting choice or model performance.
Keep exploring
Sources consulted
Corrections and transparency
If you spot incorrect or outdated information, send us a correction with the page and source we should review.
Submit a correction