What a study on looped Transformers can—and cannot—show about OpenAI

What is confirmed—and what remains unverified

The proposal about OpenAI’s next flagship model raises three questions: whether it will use a looped Transformer, whether that architecture improves mathematics and programming, and whether it complicates safety oversight. The research provided supports discussion of looped Transformers in general and of one specific experimental variant. It does not let us answer yes to the first question or confirm the other claims about OpenAI’s model.

The paper studies Fully Looped Transformer, an architecture that reuses Transformer blocks across iterations. Its authors report training experiments and evaluations of their own models. They provide no evidence that OpenAI has adopted this design, and they do not investigate an OpenAI model, a product called Astra, or a commercial deployment. Linking the paper to the company’s next model would therefore go beyond the available evidence.

What it means to reuse a Transformer in a loop

In a conventional Transformer, the network’s blocks process an input representation through a sequence of layers. In a looped model, some blocks—or a shared structure—are applied repeatedly to a state that changes over time. Rather than necessarily adding a new set of parameters for every repetition, the model reuses existing parameters. Computation can increase even when the number of parameters does not grow at the same rate.

This approach makes the number of iterations another dimension of computation. In principle, more iterations can provide additional opportunities to transform the representation, but they can also make training harder. The study does not claim that repeating blocks guarantees correct reasoning. It examines how to stabilize training and compares results in specific experimental settings.

What the study contributes—and where its limits lie

The paper describes two difficulties observed when training looped models: gradient oscillations and growth in the norm of the residual state, which it calls residual explosion. To address them, it proposes Fully Looped Transformer, with two changes: a connectivity pattern that distributes signals from previous iterations to every layer, and Attention Injection, which reuses the attention mechanism to incorporate recurrent information. According to the paper, these modifications add no learnable parameters.

In the reported experiments, the proposed variant trained stably for up to 12 iterations, while the looped baseline models collapsed at that setting. Under less demanding conditions, the authors report an improvement of up to 13.2% in average downstream-task performance. That peak result belongs to the paper’s experiments: it is not a universal improvement, a guarantee for every model, or a result attributed to OpenAI. The supplied source record does not establish that the work was peer-reviewed or independently replicated, so it should not be described as such.

Training stability and safety oversight are separate questions

The study examines training dynamics, architecture, and performance on evaluation tasks. It does not study safety audits, alignment policies, harmful-behavior detection, chain-of-thought monitoring, or release procedures. Its findings do not prove that a looped model is more opaque or harder to oversee from a safety perspective.

The work does, however, suggest a technical question for separate research: how to record and evaluate internal states and outputs at each iteration when a block is reused. But a research question is not evidence that observability gets worse. It is also important to distinguish between inspecting internal activations, evaluating visible responses, and testing behavior under safety assessments. The supplied paper compares none of these methods and proposes no alignment techniques to compensate for a supposed loss of observability.

What remains to be established about OpenAI

The evidence provided does not confirm that OpenAI has announced a looped Transformer for its next model. It also does not substantiate claims that Microsoft accidentally revealed the use of this technique, that the model will launch before the end of the year, or that additional safety measures have been introduced to compensate for reduced observability. Those points remain unverified by the available material.

A conclusive check would require primary documentation that applies to the same model and period—for example, a technical statement from OpenAI about its architecture, a Microsoft publication identifying exactly what information was disclosed, or documentation detailing evaluations and safeguards. Until such evidence is available, the responsible approach is not to turn the proposal’s hypothesis into a confirmed news claim. Inferama’s news, comparisons, and discovery sections can be consulted for further coverage, without assuming that their contents validate these claims.

Keep exploring

Corrections and transparency

If you spot incorrect or outdated information, send us a correction with the page and source we should review.