Not the number of parameters.
RAM or VRAM depending on the environment.
With the same review criteria.
Start with the work, not the model
An excellent model for code completion may not be the best choice for summarizing long documents. First define the task, data sensitivity, acceptable latency, and whether you need text, vision, or tools.
Can the data leave the device?
Do you need interactive response or batch processes?
What errors invalidate the result?
Just text or also image, audio and files?
Calculate memory with margin
The model file is only part of the consumption. The context, the cache, and the environment itself require additional space. Avoid planning to the limit and reserve margin for the system.
| Scenery | Priority | What to measure |
|---|---|---|
| Chat and writing | Perceived latency | Time to first token and consistency |
| Long documents | Helpful context | Retrieval of details and appointments |
| Programming | Correction | Passed tests and unnecessary changes |
| Batch processing | Performance | Tokens per second and energy cost |
Quantization is a trade-off, not a free shortcut
Reducing precision decreases size and can speed up inference, but the impact on quality is not uniform. Two models with the same quantized size may behave differently in your task.
The maximum window is not the useful context
The advertised limit describes how much the system can accept, not how much it will remember with the same accuracy or how much fits in your memory. The attention cache grows with the conversation and can become the dominant resource consumption even when the model weights are already loaded.
Start with the minimum context that solves the task. For long documents, compare sending the entire file with retrieving only the relevant excerpts. Measure whether it retains instructions, citations, and distant relationships; a fluent response does not prove that it used the entire input correctly.
Record weights, cache, runtime, and system margin separately.
Evaluate excerpts, overlap, and the quality of returned references.
Place facts at the beginning, middle, and end to detect attention loss.
Measure prefill and generation time at representative lengths.
License, provenance, and safety also matter
Before integrating a weights file, check the license for the model and the specific variant. “Open” may describe access to weights without granting the same rights for commercial use, redistribution, modification, or derivative training. Keep the repository, version, and license text consulted.
Treat models, templates, and downloaded files as external dependencies. Verify the origin, review formats that may execute code, limit process access, and avoid connecting tools with broad privileges during initial testing. Local execution improves data control, but it does not eliminate the risks of generated content or the supply chain.
Make a reproducible test
Saves the exact version, configuration and question set. The following example queries model information before running it with Ollama.
ollama show nombre-del-modelo
ollama run nombre-del-modelo- 01Freeze the set
Use the same 20 tasks for all variants.
- 02Register the environment
Note hardware, version, context and parameters.
- 03Blind check
Hide the model name when rating responses.