clm_5e6a0b0fWe show that when the distilled model is much too small to capture all of the knowledege in the cumbersome model, intermediate temperatures work best which strongly suggests that ign … (excerpt)
arXiv 1503.02531 · stat.ML cs.LG cs.NE
Read on arXiv · arXiv non-exclusive license
Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.
clm_5e6a0b0fWe show that when the distilled model is much too small to capture all of the knowledege in the cumbersome model, intermediate temperatures work best which strongly suggests that ign … (excerpt)
clm_04aaa6caIn this section, we investigate the effects of ensembling De ep Neural Network (DNN) acoustic models that are used in Automatic Speech Recognition (ASR).We show that the distillation strategy that … (excerpt)
res_a93fdab5This system achieves a frame accuracy of 58.9%, and a Word Error Rate (WER) of 10.9% on our development set.
clm_c307aecdThe models are randomly initi alized with different initial parameter values and we find that this creates sufficient diversity in th e trained models to allow the averaged predictions of … (excerpt)
res_d41fdfb7More than 80% of the improvement in frame classification accuracy achieved by using an ensemble of 10 models is transferred to the distilled model which is similar to the improve … (excerpt)
res_a3c4ced7The ensemble gives a smaller improvem ent on the ultimate objective of WER (on a 23K-word test set) due to the mismatch in the objecti ve function, but again, the … (excerpt)
res_1781a518With 61 specialist models, there is a 4.4% relative improvement in test accuracy overall.
res_759e6c66Table 4 show s the number of test set examples, the change in the number of examples correct at position 1 when us ing the specialist(s), and the relative percentage … (excerpt)
clm_574ef99fIn this section we demonstrate that this is a very large effect b y using far less data to fit the 85M parameters of the baseline speech model described earlier.
clm_e5a22167For really big neural networks, it can be infeasible even to train a full ensemble, but we have shown that the performance of a single really big net that has … (excerpt)
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1503.02531) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.