res_9891d249Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.
arXiv 1502.03167 · cs.LG
Read on arXiv · arXiv non-exclusive license
Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.
res_9891d249Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.
res_f994c706Using an ensemble of batchnormalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human … (excerpt)
clm_bb73ecb4We propose a new mechanism, which we call Batch Normalization , that takes a step towards reducing internal covariate shift, and in doing so dramatically accelerates the training of deep … (excerpt)
res_1ddeb463In Sec. 4.2, we apply Batch Normalization to the bestperforming ImageNet classification network, and show that we can match its performance using only 7% of the training steps, and can … (excerpt)
res_7fcf9f8dUsing an ensemble of such networks trained with Batch Normalization, we achieve the top-5 error rate that improves upon the best known results on ImageNet classification.
clm_b0cfe92eWe were interested in the comparison between the baseline and batch-normalized networks, rather than achieving the state of the art performance on MNIST (which the described architecture does not).
res_d2e349a5In a batch-normalized model, we have been able to achieve a training speedup from higher learning rates, with no ill side effects (Sec. 3.3).
res_7302089eThis led to about 1% improvements in the validation accuracy, which is consistent with the view of Batch Normalization as a regularizer (Sec. 3.4): the randomization inherent in our method … (excerpt)
res_fedd5995Inception reached the accuracy of 72.2% after 31· 106 training steps.
res_ad995ba0The Figure 3 shows, for each network, the number of training steps required to reach the same 72.2% accuracy, as well as the maximum validation accuracy reached by the network … (excerpt)
res_fac092b5BN-x5 needs 14 times fewer steps than Inception to reach the 72.2% accuracy.
res_4c023818It reaches 74.8% after6·106 steps, i.e. 5 times fewer steps than required by Inception to reach 72.2%.
res_5ff966fbIndeed, BN-x5-Sigmoid achieves the accuracy of 69.8%.
res_1433dc07Without Batch Normalization, Inception with sigmoid never achieves better than 1/1000 accuracy.
res_28f5b1d3Each network achieved its maximum accuracy after about 6· 106 training steps.
clm_db22bd1cWe have presented a novel mechanism for dramatically accelerating the training of deep networks.
clm_3b90d9d2By further increasing the learning rates, removing Dropout, and applying other modifications afforded by Batch Normalization, we reach the previous state of the art with only a small fraction of … (excerpt)
clm_fc67fa7fThe goal of Batch Normalization is to achieve a stable distribution of activation values throughout training, and in our experiments we apply it before the nonlinearity since that is where … (excerpt)
clm_1eaf0885Finally, we believe that further theoretical analysis of the algorithm would allow still more improvements and applications.
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1502.03167) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.