ByteDance releases sparse autoencoders to analyze three layers of Qwen3-30B-A3B
Image generated by the Inferama Editorial Team
01

What was released and what it studies

ByteDance Seed has released three sparse autoencoders (SAEs) trained on the output of mixture-of-experts (MoE) blocks in Qwen3-30B-A3B. Each autoencoder corresponds to a specific layer: 4, 30, or 46. The weights are organized by layer and can be downloaded as independent files.

The associated paper, titled “SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization,” was submitted to arXiv on November 5, 2026, as version 1. It is a preprint, not evidence that the article has undergone peer review. The study examines how expert contributions in the model relate to interpretable latent features.

A MoE model activates only some of its experts to process each token. Counting how often each expert is activated can reveal broad patterns, but it does not by itself indicate what part of the representation each one contributes. The paper’s method analyzes the combined output of the block and mathematically decomposes the pre-activation of each SAE feature among the active experts.

The empirical scope is limited to Qwen3-30B-A3B and the analyzed layers. The results do not demonstrate that other MoE models organize their experts in the same way, nor that interpretable labels completely or causally describe the model’s internal behavior.

02

What the three SAEs contain

Each SAE has 1,048,576 latent features and applies a TopK selection that retains up to 100 positive activations per token. The features do not form a single shared space across layers: each checkpoint was trained for the output of a specific layer, so identifiers from one layer should not be automatically treated as equivalent to those of another.

The SAE receives the unnormalized output of the MoE block, before the residual addition. Its encoder applies an affine transformation, a ReLU function, and TopK selection. The decoder combines the selected features to reconstruct an approximation of the original output. Therefore, the system serves to analyze and reconstruct internal activations; it is not a conversational model that generates responses on its own.

The weight files are in Safetensors format and contain 32-bit floating-point tensors. Each file occupies approximately 17.2 GB, in addition to the memory required to load the base model if new activations are to be captured. Long inputs may require chunked processing because the dense pre-TopK calculation consumes memory per token.

The model card publishes the fraction of unexplained variance (FVU) of the reconstruction for each layer. The value measures reconstruction error in the evaluated configuration: the lower it is, the smaller the fraction of variance left unexplained in that trial. It is not an accuracy rate, a measure of response quality, or a general model comparison.

Published weights and reconstruction figures

Qwen3-30B-A3B LayerLatent FeaturesTopK Activations per TokenPublished FVU
41,048,576Up to 10012.6%
301,048,576Up to 10025.1%
461,048,576Up to 1007.6%
03

How expert specialization is interpreted

The paper proposes measuring specialization in the feature space, rather than assigning experts to pre-defined thematic categories. Because the SAE encoder is affine and the MoE output combines the outputs of activated experts, the pre-activation of each feature can be exactly decomposed among those experts, under the work’s formulation.

By aggregating these contributions over a corpus, the analysis identifies for each expert the features whose activation it principally drives. The result is a relationship between experts and dominant features. This relationship depends on the data, layers, dominance definition, and thresholds used; it does not automatically turn a feature into a causal or universal explanation of the expert.

In the studied layers, the authors report that contributions to features are more concentrated than the router’s activation pattern and that most features have one or two dominant experts. They also describe overlaps concentrated in certain expert pairs. These are results from the analysis of Qwen3-30B-A3B, not general laws about MoE architectures.

To assign semantic descriptions to feature sets, the study uses automated interpretation with a language model. The repository indicates that the published explanations, labels, and thematic judgments were generated with GPT-5.5. They are outputs of an automated procedure and may contain errors; they do not equate to verified human annotations.

04

Training, data, and reproduction

The SAEs were trained independently using MoE outputs collected from the Olmo 3 training corpus. The described mix comprises five-eighths of Dolma 3 pretraining data, one-quarter of Dolci-Think-SFT—reasoning chains—and one-eighth of Dolci-Instruct-SFT—instruction following. The paper provides details of the training recipe; the model card reports approximately 3.2 billion tokens per SAE.

The project repository publishes code and derived data to reproduce parts of the analysis. The contribution statistics use a validation set of approximately 1.5 billion tokens, separate from the data used to train the SAEs and composed with the same source mix. The repository specifies that documents are truncated to 4,096 tokens for that processing.

Full reproduction requires considerable hardware resources. The project documentation estimates about 61 GB for Qwen3-30B-A3B in bf16 and another 8 GB for the fp32 SAE encoder of each traced layer; processing the three layers in one pass requires two 80 GB GPUs, according to that documented configuration. These figures describe the project’s procedure and do not guarantee that any implementation requires exactly the same hardware.

Some stages also depend on an OpenAI-compatible API service for automated interpretations and classifications. The repository identifies GPT-5.5 as the model used in the paper and warns that changing the model may alter those results. The published code allows reproducing specified experiments, but the repository notes that it does not include the full pipeline that gathers activating contexts and logit effects to explain features.

Analysis flow described by the project

  1. 01Capture the MoE block outputs at the selected layers of Qwen3-30B-A3B.
  2. 02Encode each output with the corresponding SAE and calculate the experts’ contributions to active features.
  3. 03Aggregate the contributions over the validation set and build dominance relationships between experts and features.
  4. 04Interpret the feature sets and evaluate labels or domains using the automated tools described in the project.
05

Practical limits and license

The published FVUs are figures attributed to the paper in the model card. Inferama has not reproduced those measurements, and the project material does not constitute independent validation. The FVU evaluates activation reconstruction in the reported trial; it does not allow concluding that the model produces more accurate, safe, or useful responses.

The relationship between an expert and a set of features provides a way to inspect internal structure, but it does not prove causality. Data selection, the threshold for defining dominance, the layer, and the interpretation method influence what is observed. Automatically generated descriptions require critical evaluation before being used as reliable explanations of behavior.

The SAE weights and the base model are published under the Apache License 2.0, according to the model card. The training data derived from Dolma 3, Dolci-Think-SFT, and Dolci-Instruct-SFT retain their own terms: the repository attributes ODC-BY v1.0 to those sets. The weights’ license does not eliminate the conditions applicable to those data or third-party services.

No official API fee is listed for these checkpoints, nor is a verified commercial offer in the documentation considered. Downloadable weights do not imply free execution: use requires storage, memory, and accelerators, and capturing activations requires loading the base model. The real cost depends on the equipment, the volume processed, and whether external services are used.

What each element allows concluding

ElementAllows assertingDoes not allow asserting
FVU per layerThe reconstruction left a fraction of variance unexplained in the reported trial.That the layer or model has a specific accuracy rate.
Dominant features per expertSome experts principally contribute to certain features in the analysis performed.That the association is causal, exclusive, or valid for other models.
Automated explanationsA language model generated labels or descriptions for features and experts.That the labels have been confirmed by human annotators.
Published weightsThe files can be downloaded and used according to their licenses and technical requirements.That an official API, published fee, or free infrastructure execution exists.
06

Keep exploring

Sources consulted

03

Corrections and transparency

If you spot incorrect or outdated information, send us a correction with the page and source we should review.

Submit a correction