One model for turning recordings into different kinds of text
NVIDIA introduced Nemotron 3.5 Transcribe as an open-weight model for working with recorded audio. It can produce transcripts in different styles, attribute segments to speakers, translate speech and answer questions about audio content. The technical report describes support for recordings up to one hour long and the option to request word-level timestamps with speaker attribution.
The model is not limited to producing a single verbatim transcript. A natural-language instruction can guide the output—for example, preserving hesitations, normalizing numbers or using domain-specific terms. This makes it possible to tailor the output to a task without retraining the model, but it does not guarantee that the requested convention will always be followed or that every word will be recognized correctly.
Language coverage depends on the task. NVIDIA reports transcription in 40 languages and translation in 31 languages to or from English; speaker attribution is described for 13 languages. A language being supported for one function does not mean all other functions have the same coverage.
Published functions and scope
The figures and capabilities in this table describe what NVIDIA has published about the model; they do not guarantee results on any particular recording.
| Function | Published scope | What to check |
|---|---|---|
| Transcription | 40 languages; recordings up to one hour long | Quality for the relevant language, accent, noise and vocabulary |
| Translation | 31 languages to or from English | Translation direction and performance for the specific language pair |
| Speaker attribution | 13 languages; up to eight speakers, according to NVIDIA’s Hugging Face publication | Speaker-separation errors, especially when voices overlap |
| Timestamps | Word-level timestamps available on request | Alignment with the audio and the timing precision required by the application |
Which model was released, and what open weights mean
The technical report describes an architecture combining a speech encoder, a frozen diarization model and a Nemotron 3.5 Lightning 30B-A3B language model. NVIDIA puts the number of active parameters at approximately 4.3 billion. That does not mean the entire model has only 4.3 billion parameters: the checkpoint name and technical description refer to a 30-billion-parameter configuration, with only some parameters active for each token.
NVIDIA says the weights are released under the OpenMDW-1.1 license and provides checkpoints in several formats, along with fine-tuning tools in NeMo Speech. “Open weights” describes access to the model weights, but does not by itself mean there are no conditions. Before integrating the model into a product, teams should review the license and the terms applying to the weights, data and tools they use.
The technical report presents the system for recorded audio, and NVIDIA’s Hugging Face publication explicitly says it is not built for live transcription. The model should therefore not be described as transcribing audio as it arrives. This distinction matters when choosing between processing a completed recording and building an interactive voice experience.
Granary v2: data and tools for adapting speech recognition
Alongside the model, NVIDIA introduced Granary v2, a dataset and curation pipeline for preparing speech examples and transcripts. NVIDIA’s publication describes a public manifest covering 463,062 hours from 16 corpora and 48 languages. Those figures apply to the published dataset manifest, not to Nemotron 3.5 Transcribe’s complete training mixture.
The pipeline produces transcript variants, task labels and quality metadata. The variants described include normalized verbatim transcription, a form with numbers written conventionally, and a version with disfluencies removed. The publication also describes processes for generating short captions, contextual terms, translations and audio-grounded question-and-answer pairs.
NVIDIA proposes combining these resources with NeMo Speech to adapt recognition to domain vocabulary, accents and writing conventions. Fine-tuning may be useful when an application depends on product names, medical terms or specific formatting. It does not remove the need for representative data and accurate labels: fine-tuning on biased examples may not reflect speech or real-world conditions.
A practical workflow for evaluating an adaptation
NVIDIA’s publication describes adaptation recipes and a medical case study. This process summarizes a cautious way to apply that idea to a specific use case, without assuming the study’s results will be repeated.
- 01Define the output you need: language, script, number formatting, disfluency handling and specialized terms.
- 02Establish a baseline using representative recordings and reviewed transcripts; separate the speakers used to tune the setup from those reserved for evaluation.
- 03First try instructions and contextual term lists. Check that suggested terms are supported by the audio rather than accepting a word just because it appears in the list.
- 04If important errors remain, evaluate fine-tuning with representative examples and verified labels.
- 05Compare the adapted model with the original on speakers and recordings not used for tuning, and review both domain-specific errors and retention on general speech.
What the published results support
NVIDIA reports favorable results on several speech-recognition and speaker-attribution benchmarks, including public evaluation sets. The technical report gives mean word error rates of 4.71% across nine sets on the English Open ASR Leaderboard and 4.62% on the multilingual evaluation. NVIDIA’s Hugging Face publication also presents results from other comparisons and metrics, whose figures depend on the benchmark and its configuration.
These figures are not a universal measure of accuracy. Word error rate depends on the recordings evaluated, the language, the reference format and normalization rules. A leaderboard position also does not show that a model will outperform another system for every accent, room, microphone or kind of conversation. The results cited here are reported by NVIDIA; the sources reviewed do not provide an independent evaluation that would confirm the company’s comparisons in general.
NVIDIA’s published medical-adaptation study offers a limited example. In an evaluation on held-out speakers, the company reports that fine-tuning reduced word error rate from 22.44% to 12.17% compared with a best-prompt baseline. It also reports a small increase in error on LibriSpeech test-clean, from 1.54% to 1.69%. This is a result for that dataset, recipe and protocol, not a guarantee of improvement for other specialties or data.
How to interpret the performance figures
A benchmark figure is meaningful alongside its test set and evaluation rules. The primary source reviewed reports the results, but does not in itself guarantee performance in other settings.
| Published figure | What it measures or describes | Interpretation limit |
|---|---|---|
| 4.71% WER | Mean reported by NVIDIA across nine sets on the English benchmark | Does not predict the error rate on every recording or in every language |
| 4.62% WER | Mean reported by NVIDIA for its multilingual evaluation | Coverage and results vary according to the languages and sets included |
| 12.17% versus 22.44% WER | Reported result for held-out speakers in a medical-adaptation case study | Depends on the corpus, prompt selection and fine-tuning procedure |
| 1.54% versus 1.69% WER | Reported LibriSpeech test-clean result before and after medical adaptation | Shows that improvement in one domain can coincide with a change in general speech |
How to assess its usefulness in an application
For meetings, dictation or analysis of recordings, combining transcription, translation and timestamps may reduce the need to chain several components together. Speaker attribution can also help reviewers check who said what, but the output should be checked against the audio when misidentification could have significant consequences.
A representative test should include the languages and accents actually used, background noise, overlapping voices and uncommon terms. It is useful to measure errors in names, numbers and domain-specific words separately from the overall average. If speaker identification matters, evaluate voice changes and simultaneous speech as well as transcription of clear conversation.
Access to the weights and fine-tuning tools offers deployment and adaptation options, but the sources reviewed do not establish total cost, latency or hardware requirements for every configuration. These factors depend on the checkpoint, quantization, server and workload. Adoption decisions should be based on an in-house test that respects the license and the conditions for handling data.
What the available evidence does not establish
The sources reviewed describe NVIDIA’s announcement, the authors’ technical report and figures published by the company. They do not establish that these results have been independently replicated or that benchmark rankings hold across all versions, workloads or configurations. The technical report is original documentation from NVIDIA; the available information is not enough to characterize it as peer-reviewed.
Nor should the Granary v2 manifest be taken to represent all of the model’s training data, or to mean that every language has the same quality across all tasks. NVIDIA’s publication distinguishes coverage by function and explains that the public manifest describes the dataset and its curation pipeline.
In short, Nemotron 3.5 Transcribe offers an open-weight model for processing recordings with multiple speech tasks and adaptation options. Its practical value depends on the specific combination of language, format and acoustic conditions. Published benchmarks provide a starting point for assessing it; the key check is an in-house evaluation using representative data and human review of important errors.
Open questions
- The sources reviewed do not provide an independent evaluation that would confirm NVIDIA’s reported performance comparisons in general.
- The available documentation does not establish equivalent results, latency or coverage for every language-and-task combination.
- Performance and deployment requirements depend on the checkpoint, configuration and conditions of use.
Keep exploring
Sources consulted
Corrections and transparency
If you spot incorrect or outdated information, send us a correction with the page and source we should review.
Submit a correction