clm_3d577b57Empirically, we show that layer normalization can substantially reduce the training time compared with previously published techniques.
arXiv 1607.06450 · stat.ML cs.LG
Read on arXiv · arXiv non-exclusive license
Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.
clm_3d577b57Empirically, we show that layer normalization can substantially reduce the training time compared with previously published techniques.
clm_2232e97fWe show that layer normalization works well for RNNs and improves both the training time and the generalization performance of several existing RNN models.
clm_8f83fd9bWe show that the normalization scalarσ can implicitly reduce learning rate and makes learning more stable.
clm_a8d3901eWe show that Riemannian metric along the magnitude of the incoming weights for the standard GLM is scaled by the norm of its input, whereas learning the gain parameters for … (excerpt)
res_6d5e6f7bAfter every 300 iterations, we compute Recall@K (R@K) values on a held out validation set and save the model whenever R@K improves.
res_c755dc68In Table 2, the test set results are reported from which we observe that layer normalization also results in improved generalization over the original model.
clm_fe4a2821The results we report are state-of-the-art for RNN embedding models, with only the structure-preserving model of Wang et al. [2016] reporting better results on this task.
res_fc0bea37We observe that applying layer normalization results both in speedup over the baseline as well as better final results after 1M iterations are performed as shown in Table 3.
clm_15132859We show how layer normalization compares with batch normalization on the well-studied permutation invariant MNIST classification problem.
clm_1f693ee4In our preliminary experiments, we observed that layer normalization offers a speedup over the baseline model without normalization, but batch normalization outperforms the other methods.
clm_34ba3bbcIn this paper, we introduced layer normalization to speed-up the training of neural networks.
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1607.06450) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.