Nemotron 3.5 Embed 8B BF16: Capabilities, Benchmarks, and Deployment Limits

What Nemotron 3.5 Embed 8B BF16 Does

Nemotron-3.5-Embed-8B-BF16 is an NVIDIA embedding model designed to retrieve documents in response to queries. It does not generate conversational answers. Instead, it converts a text query and documents made of text, images, or both into vectors that a search system can compare. A typical workflow indexes documents, stores their vectors alongside the original content, and then calculates a vector for each query to find relevant items.

The checkpoint does not generate conversational text: it transforms text queries and text, image, or mixed-modality documents into vectors for retrieval. Its Transformers, Sentence Transformers, and vLLM examples produce 4,096-dimensional vectors with L2 normalization. NVIDIA’s NeMo Retriever Embedding NIM support matrix lists 2,048 for the 8B model; that entry should not be assumed to describe the checkpoint examples’ native output. Before building an index, check the effective dimension returned by the endpoint or runtime you use.

The distinction between the checkpoint and the runtime matters in practice. A vector database typically requires stored vectors and query vectors to have compatible dimensions. Keep input preparation, task prefixes, and normalization consistent as well: if these differ between indexing and search, direct comparisons may no longer be valid.

How It Processes Text and Images

The model creates vector representations of queries and documents for retrieval tasks. Documents can consist of text, page images, or a combination of the two. This allows visual elements—such as tables, charts, and infographics—to be represented alongside accompanying text, without turning the system into an answer generator.

The model card says queries are provided as text, while documents can include text, an image, or both. Its examples use different task prefixes: “query:” for queries and “passage:” for documents. Keeping that convention and using the same runtime for indexing and search helps prevent incompatible comparisons.

The model card also describes a language tower based on Nemotron-3-Embed-8B that remained frozen during multimodal fine-tuning. NVIDIA says the two models share a text embedding space and produce identical text embeddings. This compatibility applies to text under the checkpoint conditions described; it does not authorize combining vectors with different dimensions or output configurations without checking them. The integration should keep the model, query and document preparation, normalization, and index dimensions consistent.

According to its model card, the checkpoint accepts inputs up to 32,768 tokens. Image processing uses a default longest edge of 1,120 pixels, configurable up to 1,540. These length and resolution limits describe input capabilities; they do not guarantee that every long or visually dense document will be retrieved correctly.

What the Published Benchmarks Show

NVIDIA’s model card reports results for ViDoRe V3 and MIRACL-VISION, two visual retrieval benchmarks. The reported metric is NDCG@10. The card describes ViDoRe V3 as an evaluation of visual document retrieval and MIRACL-VISION as a multilingual visual retrieval dataset. These results describe NVIDIA’s stated evaluations; they are not a universal measure of performance on every corpus.

In the published figures, Nemotron-3.5-Embed-8B-BF16 scores 60.39 on ViDoRe V3 for images, 60.65 for text, and 61.47 for image and text. Its score for images on MIRACL-VISION is 72.64. The model card also reports results for Nemotron-3.5-Embed-1B-BF16 and Qwen3-VL-Embedding-8B. These scores allow comparisons within the stated evaluation conditions, but they do not directly predict retrieval quality on a team’s own collection.

The model card says the text evaluation covered 34 languages and the image evaluation covered 18. This refers to the languages included in evaluation, not a guarantee of equivalent quality in every language, dialect, or document type. The card says the model may support additional languages, but they are not included in those evaluation sets.

The results published by NVIDIA describe the provider’s evaluations, rather than independent certification or a performance guarantee on other corpora. To decide whether the model suits a particular use case, a team can assemble its own set of queries and documents and calculate retrieval metrics using relevance judgments. Comparisons should keep conditions consistent and distinguish those local results from the provider’s scores.

Deployment Requirements and Configuration

The model card includes usage examples for Transformers, Sentence Transformers, and vLLM. NVIDIA’s NeMo Retriever Embedding NIM documentation, in turn, specifies software, CPU, and GPU requirements for its containers and lists optimized configurations by model. NIM requirements should not be treated as requirements for every way of running the checkpoint: each runtime can have different compatibility, defaults, and limits.

The optimized NIM configurations for Nemotron 3.5 Embed 1B include FP16 options, but that table does not establish the combinations supported by the 8B checkpoint. Inference precision depends on the model, runtime, and GPU. BF16 identifies the checkpoint and does not guarantee that a container uses that same inference precision; check the specific configuration before deploying the 8B variant.

The NIM matrix says kernel and precision selection can depend on the detected architecture and model configuration. A fallback configuration may run on certain compatible devices, but the documentation does not guarantee that it has the same features or performance as an optimized configuration. Being able to run a model does not mean that every combination of hardware and precision has been validated under all conditions.

The checkpoint license governs use of the model under its terms; it is not a service fee and does not establish that a particular managed endpoint is available. The cost of a self-hosted installation depends on the chosen infrastructure, its utilization, and its inference workload.

Limitations and Selection Criteria

The model is designed for retrieval, not text generation, and retrieval does not guarantee that the selected document is correct or sufficient. The model card warns that the model may not always retrieve the correct documents for a query. In applications where errors have significant consequences, check results against the original content and design controls appropriate to the use case.

The 32,768-token limit does not mean that every document of that length can be processed without loss or retrieved accurately. Inputs that exceed the checkpoint or runtime limits must be handled according to the specific implementation. Long documents may require chunking; chunks should retain enough context for a query to find the relevant section.

To compare the 8B model with the 1B variant or other alternatives, evaluate them using the same queries and documents under equivalent conditions. Also record each configuration’s output dimension, latency, and resource use. Published benchmark scores can help guide what to compare, but they cannot replace a test on the target corpus or, by themselves, justify treating one variant as better for every use case.

Nemotron-3.5-Embed-8B-BF16 may suit a retrieval workflow that needs to represent visual documents and text, provided the team can support and validate the chosen deployment. Before integrating it into an existing index, confirm dimension and configuration compatibility. If the use case involves text alone or inference workloads require tight resource limits, compare alternatives in a local evaluation rather than assuming that a larger model always offers a practical improvement.

Keep exploring

Corrections and transparency

If you spot incorrect or outdated information, send us a correction with the page and source we should review.