MearcAll papers

arXiv 1412.6980 · cs.LG

Adam: A Method for Stochastic Optimization

Diederik P. Kingma, Jimmy Ba

Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.

No reproduction recordedclm_285326f7
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.

ABSTRACT · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_bb6c4269
We propose Adam, a method for efficient stochastic optimization that only requires first-order gradients with little memory requirement.

I NTRODUCTION · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_67b6d662
Empirically, our method consistently outperforms other methods for a variety of models and datasets, as shown in section 6.

I NTRODUCTION · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_95a407d9
Overall, we show that Adam is a versatile algorithm that scales to large-scale high-dimensional machine learning problems.

I NTRODUCTION · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_5a07de6b
Using large models and datasets, we demonstrate Adam can efficiently solve practical deep learning problems.

E XPERIMENTS · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_13f1fc40
In figure 1, Adagrad outperforms SGD with Nesterov momentum by a large margin both with and without dropout noise.

E XPERIMENT : L OGISTIC REGRESSION · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_fe90e23e
Although our convergence analysis does not apply to non-convex problems, we empirically found that Adam often outperforms other methods in such cases.

E XPERIMENT : M ULTI-LAYER NEURAL NETWORKS · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_8306e738
The best results were achieved with small values of (1−β2) and bias correction; this was more apparent towards the end of optimization when gradients tends to become sparser as hidden … (excerpt)

E XPERIMENT : BIAS -CORRECTION TERM · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1412.6980) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.