Exact version
Check the model identifier, date and whether the alias can change.
DOCUMENTED BENCHMARK
DSpine Qwen3.8-27B · SWE-bench Pro throughput
Published results and conditions declared by their sources. Figures are comparable only when version, configuration, metric and date match.
01 / RESULTS
Published measurements
| Model | Organization | Result | Metric | Conditions |
|---|---|---|---|---|
| DSpine for Qwen3.8-27B | Tencent | 713,41 | Output tokens per second | Low reasoning, sampling with temperature 1, top_p 0.95 and top_k 20, concurrency eight and prefix caching. Twenty conversations per subset, replaying tool-decision prefixes. The baseline without speculation yields 241.86 tokens/s and the stated ratio is 2.95 times. Total output is divided by the full concurrent wall time: this is not per-user speed, task success rate or an independent evaluation. |
02 / INTERPRETATION
Before comparing
Equivalent configuration
Review tools, effort, number of attempts, prompt and inference budget.
Transfer to your use case
Validate the result with tasks, languages, formats and errors representative of your product.
03 / METHOD
From benchmark to reproducible decision
A public benchmark provides guidance, but does not replace an evaluation with your data, tools, budget, and error tolerance.
Identify the exact version
A moving alias and a snapshot may produce different results. Record the model, date, and provider.
Reconstruct the conditions
Prompt, reasoning, tools, number of attempts, and budget must be equivalent for comparison.
Look for uncertainty
A small difference may disappear between runs. Keep samples, dispersion, and failures, not just the average.
Check transferability
Validate whether the improvement holds across languages, formats, and real cases close to your product.
A metric is only useful when we know the task, the conditions, and the error that matters. The result begins with the design of the test.
04 / SOURCES