MearcAll papers

arXiv 2212.04356 · eess.AS cs.CL cs.LG

Robust Speech Recognition via Large-Scale Weak Supervision

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever

Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.

No reproduction recordedclm_bb71a203
When fine-tuned on standard benchmarks, this approach has improved the state of the art, especially in a low-data setting.

Introduction · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_d8422a90
We demonstrate models trained at this scale transfer well to existing datasets zeroshot, removing the need for any dataset-specific fine-tuning to achieve high-quality results.

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_8bb3524c
For an additional filtering pass, after training an initial model we aggregated information about its error rate on training Robust Speech Recognition via Large-Scale Weak Supervision 3 data sources and … (excerpt)

Data Processing · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_6213e984
As part of their analysis they concluded: “Given this result, we suspect that there is little room for a generic speech system to further improve on clean read speech without … (excerpt)

English Speech Recognition · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_8de4fbdc
Despite matching or outperforming a human on LibriSpeech dev-clean, supervised LibriSpeech models make roughly twice as many errors as a human on other datasets demonstrating their brittleness and lack of … (excerpt)

English Speech Recognition · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_8a0e41ee
The estimated robustness frontier of zero-shot Whisper models, however, includes the 95% confidence interval for this particular human.

English Speech Recognition · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e0d3a742
Although both models perform within 0.1% of each other on LibriSpeech, a zero-shot Whisper model performs much better on other datasets than expected for its LibriSpeech performance and makes 55.2% … (excerpt)

English Speech Recognition · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_30865742
Results reported in word error rate (WER) for both models after applying our text normalizer. the smallest zero-shot Whisper model, which has only 39 million parameters and a 6.7 WER … (excerpt)

English Speech Recognition · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_96159669
When compared to a human in Figure 2, the best zero-shot Whisper models roughly match their accuracy and robustness.

English Speech Recognition · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_369a54ab
For a detailed breakdown of this large improvement in robustness, Table 2 compares the performance of the best zero-shot Whisper model with a supervised LibriSpeech model that has the closest … (excerpt)

English Speech Recognition · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_1d88cf80
Despite their very close performance on the reference distribution, the zero-shot Whisper model achieves an average relative error reduction of 55.2% when evaluated on other speech recognition datasets.

English Speech Recognition · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_31751edf
Zero-shot Whisper outperforms existing models on CoV oST2 in the overall, medium, and low resource settings but still moderately underperforms on high-resource languages compared to prior directly supervised work.

Multi-lingual Speech Recognition · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_0ea59937
We achieve a new state of the art of 29.1 BLEU zero-shot without using any of the CoV oST2 training data.

Translation · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_0785d44b
Since Whisper evaluation is zero-shot, it does particularly well on the lowest resource grouping of CoV oST2, improving over mSLAM by 6.7 BLEU.

Translation · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_7cd6d9e2
NVIDIA STT models (•) perform best under low noise but are outperformed by Whisper under high noise (SNR < 10 dB).

Translation · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_bc434fbb
As an example, Welsh (CY) is an outlier with much worse than expected performance at only 13 BLEU despite supposedly having 9,000 hours of translation data.

Translation · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e762e366
The zero-shot performance of Whisper is not competitive with prior supervised work here and underperforms the supervised SOTA by 13.6%.

Translation · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_a067425b
On the 82 overlapping languages the best Whisper model achieves 80.3% accuracy.

Translation · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_ab634324
There are many models that outperform our zero-shot performance under low noise (40 dB SNR), which is unsurprising given those models are trained primarily on LibriSpeech, but all models quickly … (excerpt)

Robustness to Additive Noise · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_5c75fcac
Our model outperforms the best open source model (NVIDIA STT) on all datasets, and in most cases, commercial ASR systems as well.

Long-form Transcription · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_26b14094
Figure 7 shows the distribution of per-example WERs and aggregate WER across the 25 recordings, where the computer-assisted service has the lowest aggregate WER that is 1.15% point better than … (excerpt)

Comparison with Human Performance · page 10 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_3365a3b6
Performance improves rapidly on English speech recognition from 3,000 to 13,000 hours and then slows down noticeably between 13,000 and 54,000 hours.

Dataset Scaling · page 11 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_13cd684a
Improvements in WER follow a power-law trend for multilingual speech recognition till 54,000 hours and then deviate from this trend, improving only a further 7 points when increasing to the … (excerpt)

Dataset Scaling · page 11 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_0a70ce4f
For X→en translation, performance is practically zero when training on 7,000 hours of audio or less, and then follows a roughly log-linear improvement trend till 54,000 hours before also showing … (excerpt)

Dataset Scaling · page 11 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_09c4864e
The general trend across tasks of diminishing returns when moving from 54,000 hours to our full dataset size of 680,000 hours could suggest that the current best Whisper models are … (excerpt)

Dataset Scaling · page 11 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_1a680cf4
However, multilingual and multitask models benefit more from scale and eventually outperform models trained on English data only.

Multitask and Multilingual Transfer · page 12 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_c1404545
95% bootstrap estimate confidence intervals are shown. models scale better and for our largest experiments outperform their English-only counterparts demonstrating positive transfer from other tasks.

Multitask and Multilingual Transfer · page 12 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_300a1e8f
For our largest experiments, joint models also slightly outperform English-only models even when not adjusting for compute spent per task.

Multitask and Multilingual Transfer · page 12 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_4798ee05
We start with temperature 0, i.e. always selecting the tokens with the highest probability, and increase the temperature by 0.2 up to 1.0 when either the average log probability over … (excerpt)

Strategies for Reliable Long-form Transcription · page 12 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_138455bb
Providing the transcribed text from the preceding window as previous-text conditioning when the applied temperature is below 0.5 further improves the performance.

Strategies for Reliable Long-form Transcription · page 12 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_146ae8df
We achieve our results without the need for the self-supervision and self-training techniques that have been a mainstay of recent large-scale speech recognition work and demonstrate how simply training on … (excerpt)

Conclusion · page 14 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_becdeeda
English transcription WER (%) with beam search and temperature fallback Robust Speech Recognition via Large-Scale Weak Supervision 23

Model · page 22 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_77a6d43b
WER (%) on V oxPopuli Robust Speech Recognition via Large-Scale Weak Supervision 24

Model · page 23 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_6b1db2d0
WER (%) on Fleurs Robust Speech Recognition via Large-Scale Weak Supervision 25

Model · page 24 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_7b340428
BLEU scores on Fleurs Robust Speech Recognition via Large-Scale Weak Supervision 26

Model · page 25 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_98137317
Long-form English transcription WER (%) Robust Speech Recognition via Large-Scale Weak Supervision 27

Model · page 26 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2212.04356) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.