By Paul Derrington, Jr.
CEO | Chief Engineer | Research Engineer | Editor, D-CSIL
5 min read
Key Takeaways
NVIDIA released Nemotron 3.5 Transcribe on October 28, an open-weight model that turns a recording into a transcript, speaker labels, a translation, or answers to questions about the audio, all from one checkpoint. It transcribes 40 languages, translates 31 of them to or from English, and labels up to eight speakers in recordings up to an hour long. NVIDIA reports it leads open-weight models on transcription accuracy and speaker attribution, and independent Artificial Analysis results tie it for first among open-weight models. It works on recorded audio only, not live streams, and language support differs by task.
Start with one recording and a plain instruction
Take a meeting recording and give Transcribe one instruction in plain language. NVIDIA's examples are direct: “Transcribe this recording and label each speaker,” “Translate this audio into English,” or “What decisions were made in this meeting?” The same checkpoint produces a different output for each instruction, with no retraining in between, and it can return word-level timestamps so each line can be matched back to the audio.
The prompt can also carry the vocabulary the recording needs. NVIDIA's healthcare example reads: “Transcribe this medical discussion in the original language. Possible terms: HbA1c, metformin. Use these terms only when supported by the audio. Preserve standard medical spelling and write numbers in conventional written form.” That “only when supported by the audio” matters: a term list is a hint, not permission to invent words that were not said.
Language support is not uniform. Transcription covers 40 languages, translation covers 31 of them to or from English, and speaker attribution covers 13. Audio question answering was trained in English. NVIDIA's own advice stands: spot-check names, speaker changes, and answers against the recording.
How one model does four jobs
Transcribe pairs a speech encoder with a hybrid Mamba-Transformer mixture-of-experts language model, NVIDIA's Nemotron 3.5 Lightning 30B-A3B. The full model has 30 billion parameters, with about 4 billion active per token, which is why NVIDIA positions it as a single-GPU model rather than a datacenter-only one.
Speaker labels are built in, not bolted on afterward. Speaker cues from NVIDIA's frozen Nemotron 3 Diarization model, built on Sortformer, reach the language model alongside the acoustic features, so the model tags each turn while it writes. A lightweight CTC adapter then aligns the generated transcript to the audio, which is what produces speaker-attributed word timestamps even for long, heavily overlapping recordings.
Context does measurable work. Given a list of expected entities, NVIDIA reports Transcribe cuts named-entity errors six-fold on ContextASR-Bench. On speed, NVIDIA's benchmark processed 100 hours of audio in under three minutes on one RTX PRO 6000 Blackwell Server Edition GPU using the forthcoming NVFP4 quantized version, measured on 30-second clips at high concurrency. Treat that as a throughput benchmark under those conditions, not a promise for a single long file on lesser hardware.
The numbers behind the lead, with the caveats attached
On the Hugging Face Open ASR Leaderboard, NVIDIA reports Transcribe first among open-weight models in English at 5.33% average word error rate, with API models excluded from that ranking. The same comparison gives 4.62% mean word error rate across seven languages on the multilingual track and 10.98% on the long-form track, again the lowest among the evaluated open-weight models. Lower word error rate means fewer transcription errors.
The independent check comes from Artificial Analysis, which ties Transcribe for first among open-weight models at 2.8% AA-WER, against about 4 billion active parameters versus 24 billion for the model it ties. On AA-AgentTalk, the held-out voice-agent speech set that makes up half of that index, Transcribe scores 1.8% word error rate versus 2.3% for the next-best open-weight model.
Speaker attribution is where the lead is widest. Transcribe posts the lowest diarization error rate of 14 systems on David AI's 21-language benchmark, 18.26%, just ahead of NVIDIA's own dedicated diarization model at 18.38%. On VoiceArena's English full-session benchmark it ranks first at 32.4% cpWER, a word-error score counted after matching speaker labels, and drops the fewest words: deletions are 8.32% of reference words versus a median of about 31%. These are particular tests, and the leaderboard figures are NVIDIA's reported comparisons except for Artificial Analysis. Test the model on your own recordings before relying on it.
The medical example shows what adapting it actually takes
NVIDIA's case study uses Eka Care's open benchmark of recordings from Indian doctors and medical students: 3,619 clips, with no designated training split. On the full test set, naming English and the Latin alphabet in the prompt cut word error rate from 30.8% to 23.8%, and fine-tuning brought it to 11.8%. The cleaner test splits the benchmark by speaker, holding out 2,764 clips decoded once: there, fine-tuning cut word error rate from 22.44% for the best-prompt baseline to 12.17%, a 45.8% relative reduction, with keyword word error rate falling from 9.2% to 6.5%. On the same held-out speakers and scorer, the adapted model beat an open medical-specialist model at 13.79% and a widely used general-purpose speech model at 14.92%.
The adaptation took about 14 GPU-hours on one NVIDIA GB300, using about 153 hours of synthetic speech plus 29 hours of accented English from Common Voice and 63 hours of medical recordings from MultiMed. There is a trade-off to note: general English on LibriSpeech test-clean slipped slightly, from 1.54% to 1.69% word error rate, after medical fine-tuning. NVIDIA's study advises keeping at least 100 real, labeled clips for checkpoint selection and evaluating the entity types an application depends on alongside general-speech retention.
Alongside the model, NVIDIA released Granary v2, the data curation pipeline behind it. Its public manifest covers 463,062 hours across 48 languages and 16 source corpora, up from 25 languages in Granary v1, with three transcript forms for each recording. Download source audio separately under each corpus's original license. The model itself is released under the OpenMDW-1.1 license, with fine-tuning tools in NeMo Speech, a demo on Hugging Face and ModelScope, hosted endpoints from Baseten, DeepInfra, and Simplismart, and vLLM serving today. The hard limit stays: Transcribe is built for recorded audio, not live streaming.