clm_bb71a203When fine-tuned on standard benchmarks, this approach has improved the state of the art, especially in a low-data setting.
arXiv 2212.04356 · eess.AS cs.CL cs.LG
Read on arXiv · arXiv non-exclusive license
Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.
clm_bb71a203When fine-tuned on standard benchmarks, this approach has improved the state of the art, especially in a low-data setting.
clm_d8422a90We demonstrate models trained at this scale transfer well to existing datasets zeroshot, removing the need for any dataset-specific fine-tuning to achieve high-quality results.
res_8bb3524cFor an additional filtering pass, after training an initial model we aggregated information about its error rate on training Robust Speech Recognition via Large-Scale Weak Supervision 3 data sources and … (excerpt)
res_6213e984As part of their analysis they concluded: “Given this result, we suspect that there is little room for a generic speech system to further improve on clean read speech without … (excerpt)
clm_8de4fbdcDespite matching or outperforming a human on LibriSpeech dev-clean, supervised LibriSpeech models make roughly twice as many errors as a human on other datasets demonstrating their brittleness and lack of … (excerpt)
res_8a0e41eeThe estimated robustness frontier of zero-shot Whisper models, however, includes the 95% confidence interval for this particular human.
res_e0d3a742Although both models perform within 0.1% of each other on LibriSpeech, a zero-shot Whisper model performs much better on other datasets than expected for its LibriSpeech performance and makes 55.2% … (excerpt)
res_30865742Results reported in word error rate (WER) for both models after applying our text normalizer. the smallest zero-shot Whisper model, which has only 39 million parameters and a 6.7 WER … (excerpt)
res_96159669When compared to a human in Figure 2, the best zero-shot Whisper models roughly match their accuracy and robustness.
res_369a54abFor a detailed breakdown of this large improvement in robustness, Table 2 compares the performance of the best zero-shot Whisper model with a supervised LibriSpeech model that has the closest … (excerpt)
res_1d88cf80Despite their very close performance on the reference distribution, the zero-shot Whisper model achieves an average relative error reduction of 55.2% when evaluated on other speech recognition datasets.
clm_31751edfZero-shot Whisper outperforms existing models on CoV oST2 in the overall, medium, and low resource settings but still moderately underperforms on high-resource languages compared to prior directly supervised work.
res_0ea59937We achieve a new state of the art of 29.1 BLEU zero-shot without using any of the CoV oST2 training data.
res_0785d44bSince Whisper evaluation is zero-shot, it does particularly well on the lowest resource grouping of CoV oST2, improving over mSLAM by 6.7 BLEU.
res_7cd6d9e2NVIDIA STT models (•) perform best under low noise but are outperformed by Whisper under high noise (SNR < 10 dB).
res_bc434fbbAs an example, Welsh (CY) is an outlier with much worse than expected performance at only 13 BLEU despite supposedly having 9,000 hours of translation data.
res_e762e366The zero-shot performance of Whisper is not competitive with prior supervised work here and underperforms the supervised SOTA by 13.6%.
res_a067425bOn the 82 overlapping languages the best Whisper model achieves 80.3% accuracy.
res_ab634324There are many models that outperform our zero-shot performance under low noise (40 dB SNR), which is unsurprising given those models are trained primarily on LibriSpeech, but all models quickly … (excerpt)
clm_5c75fcacOur model outperforms the best open source model (NVIDIA STT) on all datasets, and in most cases, commercial ASR systems as well.
res_26b14094Figure 7 shows the distribution of per-example WERs and aggregate WER across the 25 recordings, where the computer-assisted service has the lowest aggregate WER that is 1.15% point better than … (excerpt)
res_3365a3b6Performance improves rapidly on English speech recognition from 3,000 to 13,000 hours and then slows down noticeably between 13,000 and 54,000 hours.
res_13cd684aImprovements in WER follow a power-law trend for multilingual speech recognition till 54,000 hours and then deviate from this trend, improving only a further 7 points when increasing to the … (excerpt)
res_0a70ce4fFor X→en translation, performance is practically zero when training on 7,000 hours of audio or less, and then follows a roughly log-linear improvement trend till 54,000 hours before also showing … (excerpt)
res_09c4864eThe general trend across tasks of diminishing returns when moving from 54,000 hours to our full dataset size of 680,000 hours could suggest that the current best Whisper models are … (excerpt)
clm_1a680cf4However, multilingual and multitask models benefit more from scale and eventually outperform models trained on English data only.
res_c140454595% bootstrap estimate confidence intervals are shown. models scale better and for our largest experiments outperform their English-only counterparts demonstrating positive transfer from other tasks.
clm_300a1e8fFor our largest experiments, joint models also slightly outperform English-only models even when not adjusting for compute spent per task.
res_4798ee05We start with temperature 0, i.e. always selecting the tokens with the highest probability, and increase the temperature by 0.2 up to 1.0 when either the average log probability over … (excerpt)
res_138455bbProviding the transcribed text from the preceding window as previous-text conditioning when the applied temperature is below 0.5 further improves the performance.
clm_146ae8dfWe achieve our results without the need for the self-supervision and self-training techniques that have been a mainstay of recent large-scale speech recognition work and demonstrate how simply training on … (excerpt)
res_becdeedaEnglish transcription WER (%) with beam search and temperature fallback Robust Speech Recognition via Large-Scale Weak Supervision 23
res_77a6d43bWER (%) on V oxPopuli Robust Speech Recognition via Large-Scale Weak Supervision 24
res_6b1db2d0WER (%) on Fleurs Robust Speech Recognition via Large-Scale Weak Supervision 25
res_7b340428BLEU scores on Fleurs Robust Speech Recognition via Large-Scale Weak Supervision 26
res_98137317Long-form English transcription WER (%) Robust Speech Recognition via Large-Scale Weak Supervision 27
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2212.04356) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.