clm_285326f7We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.
arXiv 1412.6980 · cs.LG
Read on arXiv · arXiv non-exclusive license
Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.
clm_285326f7We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.
clm_bb6c4269We propose Adam, a method for efficient stochastic optimization that only requires first-order gradients with little memory requirement.
res_67b6d662Empirically, our method consistently outperforms other methods for a variety of models and datasets, as shown in section 6.
clm_95a407d9Overall, we show that Adam is a versatile algorithm that scales to large-scale high-dimensional machine learning problems.
clm_5a07de6bUsing large models and datasets, we demonstrate Adam can efficiently solve practical deep learning problems.
res_13f1fc40In figure 1, Adagrad outperforms SGD with Nesterov momentum by a large margin both with and without dropout noise.
clm_fe90e23eAlthough our convergence analysis does not apply to non-convex problems, we empirically found that Adam often outperforms other methods in such cases.
res_8306e738The best results were achieved with small values of (1−β2) and bias correction; this was more apparent towards the end of optimization when gradients tends to become sparser as hidden … (excerpt)
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1412.6980) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.