Exact version
Check the model identifier, date and whether the alias can change.
Published results and conditions declared by their sources. Figures are comparable only when version, configuration, metric and date match.
| Model | Organization | Result | Metric | Conditions |
|---|---|---|---|---|
| GPT‑5.6 Sol | OpenAI | 88,8 % | Accuracy | With the published harness and effort setting. |
| GPT‑5.6 Terra | OpenAI | 87,4 % | Accuracy | With the published harness and effort setting. |
| GPT‑5.6 Luna | OpenAI | 84,7 % | Accuracy | With the published harness and effort setting. |
| Gemini 3.1 Pro | 70,7 % | Accuracy | Figure reproduced in OpenAI's comparison table. | |
| DeepSeek V4.1 Flash | DeepSeek | 90,6 % | Accuracy | Historical result from September 10, 2026, not re-evaluated for API revision 1017. |
| Ripple-65B Preview | Ricursive Intelligence | 80,5 / 100 | Score | Ricursive · assistant, Terminus 2; not reproduced by Inferama. |
| LongCat-2.5 | Meituan | 88,4 / 100 | Score | Meituan · Claude Code v2.1.280 · 512K · average of three runs with reasoning enabled. Not reproduced by Inferama. |
| Recursive Flint | Recursive | 54.9% | Higher is better | Recursive · mean pass@1 over at least five attempts · temperature 1.0, top-p 0.95, top-k 20. Terminus 2 for Terminal-Bench; mini-swe-agent for SWE-bench. Provider result, not reproduced by Inferama; do not extrapolate between precisions. |
Check the model identifier, date and whether the alias can change.
Review tools, effort, number of attempts, prompt and inference budget.
Validate the result with tasks, languages, formats and errors representative of your product.
A public benchmark provides guidance, but does not replace an evaluation with your data, tools, budget, and error tolerance.
A moving alias and a snapshot may produce different results. Record the model, date, and provider.
Prompt, reasoning, tools, number of attempts, and budget must be equivalent for comparison.
A small difference may disappear between runs. Keep samples, dispersion, and failures, not just the average.
Validate whether the improvement holds across languages, formats, and real cases close to your product.
A metric is only useful when we know the task, the conditions, and the error that matters. The result begins with the design of the test.