Speech model trained on Monsoon data cuts conversational errors
A speech recognition model trained further on Monsoon data cut meaning-related transcription error rates by 20–44 percentage points against a same-design model trained on public data.
Edited by Boris Djordjevic · Paperfoot
The Monsoon data-collection framework gathered unscripted speech in 50 languages; researchers analyzed seven collections of 3,500 hours each. It paired contributors for conversations, screened recordings automatically and put transcripts through five levels of human review. It also selected segments for variety in words and sounds. At equal data volume, its quality stages reduced semantic word error rate—a measure of meaning-related transcription errors—by 10–39% relative to a conventional collection protocol.
Whisper, a speech recognition model, received further training on Monsoon data. It reduced that error rate by 20–44 percentage points on conversational tests against a model with the same design trained on public speech collections. It performed comparably on the comparison model’s own tests. Gains also transferred to a second model design and grew with data volume. The authors released the framework, quality-control models and further-trained models.
Why it matters
The benchmark results showed that changing the speech training data could improve conversational transcription while keeping the model design the same.
Caveats
The abstract does not name the conversational benchmarks or report variation across runs, limiting assessment of the gains’ breadth and consistency. The work is a preprint, not yet peer-reviewed.
In this paper
low-resource spontaneous speech collection for ASR
- Methods
- IndicWhisper (baseline)MAI-Transcribe-2 (baseline)Gemini 3.1 Pro (baseline)ElevenLabs Scribe v2 (baseline)Muse Voice Transcribe (baseline)GPT Realtime Whisper (baseline)Whisper medium (evaluated)SraVaani-1.0 (evaluated)Gemini 3.7 FlashBGE-smallXLS-R 300MPANNs
- Datasets
- CommonVoice (evaluated)FLEURS (evaluated)GramVaani (evaluated)IndicTTS (evaluated)Kathbath (evaluated)MUCS (evaluated)IndicVoices (evaluated)Voice of India (evaluated)Vistaar (evaluated)Monsoon (this paper)
Results
- Whisper medium, error, on IndicVoices, fine-tuned on Monsoon: 10–16%; Whisper medium, error, on IndicVoices, in Telugu: 1.6 points
Fine-tuning on Monsoon reduces Whisper medium’s error from 57–93% to 10–16%, after which it ranks second in five of the six languages, behind only MAI-Transcribe-2, and first in Telugu, where it leads MAI-Transcribe-2 by 1.6 points, and it is ahead of Gemini 3.1 Pro, ElevenLabs Scribe v2, Muse Voice Transcribe, and GPT Realtime Whisper in every language …
4.4 How does Monsoon compare with commercial systems?
- Whisper medium, error, on IndicVoices, averaged over the six languages: 13% lower
Averaged over the six languages, its error is 13% lower than that of Gemini 3.1 Pro, the second-best commercial system.
4.4 How does Monsoon compare with commercial systems?
- Whisper medium, semantic WER: 20 to 44 points
Whisper medium fine-tuned on Monsoon matches the architecture-matched IndicWhisper on the latter’s own test sets and outperforms it by 20 to 44 points of semantic WER on independently collected spontaneous speech.
1 Introduction
- IndicWhisper, error, on IndicVoices: above 32%
On both IndicVoices and Voice of India, every IndicWhisper point lies above 32% and every Monsoon point below 17%, and within each language the gap is at least 20 points.
4.1 Does Monsoon generalise beyond its own collection?
- SraVaani-1.0, semantic WER, on IndicVoices, across four representative languages, Hindi, Bengali, Gujarati, and Telugu: 13 to 28% relative
Continual fine-tuning on Monsoon reduces its semantic WER on IndicVoices by 13 to 28% relative across four representative languages, Hindi, Bengali, Gujarati, and Telugu, with the largest gain in Gujarati.
4.3 Do the gains transfer across architectures?
- Whisper medium, error, on IndicVoices, from 1,000 hours to the full corpus: 11 to 14% relative; Whisper medium, error, on Vistaar, from 1,000 hours to the full corpus: 21 to 25%
Error falls monotonically on all six curves: from 1,000 hours to the full corpus, it drops by 11 to 14% relative on IndicVoices and by 21 to 25% on the Vistaar sets, so the gain from additional data is roughly twice as large on the Vistaar sets.
4.5 Does accuracy keep improving with more data?
What it shows
- Monsoon's quality stages reduce semantic word error rate (WER) by 10 to 39% relative over a conventional protocol at equal volume and with the same elicitation.
At equal volume and with the same elicitation, the framework’s quality stages reduce error by 10 to 39% relative over a conventional protocol.
1 Introduction
- Whisper medium fine-tuned on Monsoon matches IndicWhisper on IndicWhisper's test sets and outperforms it by 20 to 44 points of semantic WER on independently collected spontaneous speech.
Whisper medium fine-tuned on Monsoon matches the architecture-matched IndicWhisper on the latter’s own test sets and outperforms it by 20 to 44 points of semantic WER on independently collected spontaneous speech.
1 Introduction
- Classifiers fine-tuned from XLS-R 300M on over 20,000 hours of labelled audio verify language and speaker gender with over 95% held-out accuracy.
Classifiers fine-tuned from XLS-R 300M on over 20,000 hours of labelled audio verify language and speaker gender with over 95% held-out accuracy, and PANNs rejects recordings in which any recorded-media class, such as music, exceeds probability 0.5.
3 The Monsoon Framework
- On IndicVoices, Whisper medium fine-tuned on Monsoon leads MAI-Transcribe-2 by 1.6 points in Telugu and ranks ahead of multiple commercial systems in every evaluated language.
Fine-tuning on Monsoon reduces Whisper medium’s error from 57–93% to 10–16%, after which it ranks second in five of the six languages, behind only MAI-Transcribe-2, and first in Telugu, where it leads MAI-Transcribe-2 by 1.6 points, and it is ahead of Gemini 3.1 Pro, ElevenLabs Scribe v2, Muse Voice Transcribe, and GPT Realtime Whisper in every language …
4.4 How does Monsoon compare with commercial systems?
Quoted word for word from the full text on arXiv.
The paper
Monsoon: Scalable Spontaneous Speech Collection for Low-Resource ASR in 50 Languages
arXiv, Oct 19, 2026, Open access, Preprint, not peer-reviewed
doi.org/10.48550/arxiv.2610.26776
arXiv:2610.26776v1 in cs.CL
Comments: Submitted to ICASSP 2027
Cite as BibTeX
@misc{banga2026monsoon,
title = {Monsoon: Scalable Spontaneous Speech Collection for Low-Resource ASR in 50 Languages},
author = {Banga, Shobhit and Dhir, Manas and Pareek, Aaditya and Walecha, Amritansh and Singh, Bhaskar and Sidh, Hanuman and Kaur, Manmeet and Jain, Sagar and Paul, Supriya and Chhabra, Vanshika and Singh, Utkarsh},
year = {2026},
eprint = {2610.26776},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
doi = {10.48550/arxiv.2610.26776},
url = {https://arxiv.org/abs/2610.26776}
}