MearcAll papers

arXiv 1409.3215 · cs.CL cs.LG

Sequence to Sequence Learning with Neural Networks

Ilya Sutskever, Oriol Vinyals, Quoc V. Le

Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.

No reproduction recordedclm_765de187
In this paper, we present a general end-to-end ap proach to sequence learning that makes minimal assumptions on the sequence structure.

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_3a7f25aa
Our main result is that on an English to French translation task from the WMT’14 dataset, the translationsproduced by the LSTM achieve a BLEU score of 34.8 on the entire … (excerpt)

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_516e4bc4
For comparison, a phrase- based SMT system achieves a BLEU score of 33.3 on the same dataset.

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_7a1ff2e6
When we used the LSTM to rerank the 1000 hypotheses produced by the aforementione d SMT system, its BLEU score increases to 36.5, which is close to the previous b … (excerpt)

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_61f0c1ab
In this paper, we show that a straig htforward application of the Long Short-Term Memory (LSTM) architecture [16] can solve gener al sequence to sequence problems.

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_b12c9d87
Our approach is closely relat ed to Kalchbrenner and Blunsom [18] who were the first to map the entire input sentence to vector, and is related to Cho et al. … (excerpt)

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_3bc410ca
On the WMT’14 En glish to French translation task, we obtained a BLEU score of 34.81 by directly extracting translations from an ensemble of 5 de ep LSTMs (with 384M … (excerpt)

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_1fcebab2
The 34.81 BLEU score was achieved by an LSTM with a vocabulary of 80k words, so the score was penalized whenever the reference translation contained a word not cov ered … (excerpt)

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_16450f6a
This result shows that a relatively unoptimized small-vocabulary neural net work architecture which has much room for improvement outperforms a phrase-based SMT system.

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_73b01852
By doing so, we obtained a BLEU score of 36.5, which improves the baseline by 3.2 BLEU points and is close to the previous best published result on this task … (excerpt)

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_cc2aefdc
Second, we found that deep LSTMs significantly outperformed shallow LSTMs, so we chose an LSTM with four layers.

The model · page 3 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_ac4570bd
The resulting implementation achieved a speed of 6,300 (both English and French) words per second with a minibatch size of 128.

Parallelization · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_8b850891
While the decoded translations of the LSTM ensemble do not outperform the best WMT’14 system, it is the first time that a pure neural translation system outperforms a phrase-based SMT … (excerpt)

Experimental Results · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_4a2a3308
There is no degradation on sentences with less than 35 words, there is only a minor degradation on the longest sentences.

Model Analysis · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_244e4735
In this work, we showed that a large deep LSTM, that has a limit ed vocabulary and that makes almost no assumption about problem structure can outperform a standard SMT-based … (excerpt)

Conclusion · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_c2329a95
We were surprised by the extent of the improvement obtained b y reversing the words in the source sentences.

Conclusion · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_9d9504ec
Most importantly, we demonstrated that a simple, straightf orward and a relatively unoptimized approach can outperform an SMT system, so further work will lik ely lead to even greater translation … (excerpt)

Conclusion · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1409.3215) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.