Infrastructure, chips, inference and models that run on your own devices.
27 articles
Following an 11-day pause triggered by security incidents, OpenAI has decided to eliminate real-time internet access for its most advanced models, replacing it with a simulated environment using cached web content.
Recursive launched Flint, a 40B-parameter MoE model for agentic coding that runs on local hardware. Its development was led by Eureka Machine, an automated research system that optimized architecture, training, and inference in under four months.
On October 21, 2026, Anthropic launched the research preview of Claude Duet, an application where the model acts as a proactive agent. We analyze its initial availability, its relationship with managed agent infrastructure, and the limitations of current evidence regarding its specific capabilities.
Four NVIDIA models were made public on October 20, 2026: two embedding systems, a multimodal reranker, and an image editing model for board inspection. All operate under the OpenMDW 1.1 license and require self-hosted infrastructure.
DeepSeek has released its internal inference framework, which it already uses in production with V4.1-Flash. The release may facilitate self-hosted deployments, but the available sources do not independently confirm its performance advantages, the scope of hardware support, or an effective reduction in NVIDIA dependency.
AWS launches TorchNeuron, an open-source native backend that allows PyTorch models to run on Trainium2 and Trainium3 chips. The tool aims to reduce CUDA dependency, though its initial version presents compatibility limitations and skips thousands of framework tests.
Inception Labs has introduced Mercury 3, a diffusion-based language model that the company says can generate 1,068 tokens per second and handle a one-million-token context. We explain the architecture, examine the company’s benchmark claims, and distinguish reported specifications from what has been independently verified.
vLLM's disaggregated inference architecture now incorporates a bidirectional KV cache transfer mechanism that reuses context across turns, significantly improving time to first token for agentic workloads.
Nemotron 3.5 Super VL is designed to interpret documents, charts, images, and video while supporting long-running tasks with tools. Available information confirms the launch and model scale, but it is insufficient to establish independent performance or the specific NVIDIA hardware required for each deployment.
The new model combines reasoning, tool use, and understanding of text, images, documents, and video. NVIDIA says it activates 12 billion parameters per token and offers efficiency improvements, but the published results are not enough to establish an independent comparison or a universal inference cost.
Researchers present a method to project the outputs of biomolecular diffusion models, correcting physical defects such as chain clashes or distorted bonds without the need to retrain the model.
A recent preprint proposes a system that dynamically adjusts the distribution of workers between prefill and decode phases to prevent SLO violations without requiring additional GPUs.