Infrastructure, chips, inference and models that run on your own devices.
31 articles
The benchmark draws on tasks based on real changes to Megatron-LM, vLLM, and verl. In the preprint, the best result among six frontier models was solving 38.9% of the tasks.
Recursive Superintelligence has released Flint, an open-weight model with 39.8 billion parameters designed for local code inference. With only 2.5 billion active parameters per token, the company claims speeds exceeding 125 tokens per second on Apple Silicon laptops.
LFM3 brings together text, vision, audio, and decision models built from a shared foundation. Liquid AI says the model weights are available and each model can be fine-tuned on a single GPU; the published performance figures are the company’s own evaluations, not independent verification.
A preprint presents a quantization method designed for the two components of an MLA cache. Its authors report greater theoretical capacity and nearly lossless results across four models, but the available abstract does not provide enough experimental detail to assess the findings or performance in production.
Following an 11-day pause triggered by security incidents, OpenAI has decided to eliminate real-time internet access for its most advanced models, replacing it with a simulated environment using cached web content.
Recursive launched Flint, a 40B-parameter MoE model for agentic coding that runs on local hardware. Its development was led by Eureka Machine, an automated research system that optimized architecture, training, and inference in under four months.
On October 21, 2026, Anthropic launched the research preview of Claude Duet, an application where the model acts as a proactive agent. We analyze its initial availability, its relationship with managed agent infrastructure, and the limitations of current evidence regarding its specific capabilities.
Four NVIDIA models were made public on October 20, 2026: two embedding systems, a multimodal reranker, and an image editing model for board inspection. All operate under the OpenMDW 1.1 license and require self-hosted infrastructure.
DeepSeek has released its internal inference framework, which it already uses in production with V4.1-Flash. The release may facilitate self-hosted deployments, but the available sources do not independently confirm its performance advantages, the scope of hardware support, or an effective reduction in NVIDIA dependency.
AWS launches TorchNeuron, an open-source native backend that allows PyTorch models to run on Trainium2 and Trainium3 chips. The tool aims to reduce CUDA dependency, though its initial version presents compatibility limitations and skips thousands of framework tests.
Inception Labs has introduced Mercury 3, a diffusion-based language model that the company says can generate 1,068 tokens per second and handle a one-million-token context. We explain the architecture, examine the company’s benchmark claims, and distinguish reported specifications from what has been independently verified.
vLLM's disaggregated inference architecture now incorporates a bidirectional KV cache transfer mechanism that reuses context across turns, significantly improving time to first token for agentic workloads.