Exact version
Check the model identifier, date and whether the alias can change.
DOCUMENTED BENCHMARK
GIFT-Eval CRPS
Published results and conditions declared by their sources. Figures are comparable only when version, configuration, metric and date match.
01 / RESULTS
Published measurements
| Model | Organization | Result | Metric | Conditions |
|---|---|---|---|---|
| Timestral 5M | Mistral AI | 0,479 | Lower is better | Metrics reported by Mistral for each checkpoint, not reproduced by Inferama. Lower MASE and CRPS are better; higher fev-bench skill scores are better. These are not accuracy percentages or conversational tests. The technical report is still pending. |
| Timestral 28M | Mistral AI | 0,461 | Lower is better | Metrics reported by Mistral for each checkpoint, not reproduced by Inferama. Lower MASE and CRPS are better; higher fev-bench skill scores are better. These are not accuracy percentages or conversational tests. The technical report is still pending. |
| Timestral 123M | Mistral AI | 0,448 | Lower is better | Metrics reported by Mistral for each checkpoint, not reproduced by Inferama. Lower MASE and CRPS are better; higher fev-bench skill scores are better. These are not accuracy percentages or conversational tests. The technical report is still pending. |
| Timestral 209M | Mistral AI | 0,446 | Lower is better | Metrics reported by Mistral for each checkpoint, not reproduced by Inferama. Lower MASE and CRPS are better; higher fev-bench skill scores are better. These are not accuracy percentages or conversational tests. The technical report is still pending. |
| Granite Timeseries Ensemble R1 | IBM | 0,4707 | Normalized CRPS/MWQL | IBM · linear_pool · 97 configurations · standings on September 28, 2026 · lower is better. |
| Granite Timeseries Ensemble R1 | IBM | 0,4700 | Normalized CRPS/MWQL | IBM · iqr_weighted · 97 configurations · standings on September 28, 2026 · lower is better. |
02 / INTERPRETATION
Before comparing
Equivalent configuration
Review tools, effort, number of attempts, prompt and inference budget.
Transfer to your use case
Validate the result with tasks, languages, formats and errors representative of your product.
03 / METHOD
From benchmark to reproducible decision
A public benchmark provides guidance, but does not replace an evaluation with your data, tools, budget, and error tolerance.
Identify the exact version
A moving alias and a snapshot may produce different results. Record the model, date, and provider.
Reconstruct the conditions
Prompt, reasoning, tools, number of attempts, and budget must be equivalent for comparison.
Look for uncertainty
A small difference may disappear between runs. Keep samples, dispersion, and failures, not just the average.
Check transferability
Validate whether the improvement holds across languages, formats, and real cases close to your product.
A metric is only useful when we know the task, the conditions, and the error that matters. The result begins with the design of the test.
04 / SOURCES