MearcAll papers

arXiv 1409.0473 · cs.CL cs.LG cs.NE

Neural Machine Translation by Jointly Learning to Align and Translate

Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio

Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.

No reproduction recordedclm_d82d1000
With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation.

ABSTRACT · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_14a57aeb
We show this allows a model to cope better with long sentences.

I NTRODUCTION · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_f5c7b3aa
In this paper, we show that the proposed approach of jointly learning to align and translate achieves significantly improved translation performance over the basic encoder–decoder approach.

I NTRODUCTION · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_e32e0d22
Here, we describe briefly the underlying framework, called RNN Encoder–Decoder, proposed by Cho et al. (2014a) and Sutskever et al. (2014) upon which we build a novel architecture that learns … (excerpt)

RNN E NCODER –D ECODER · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_4bdf2b4b
1 We mean by the state-of-the-art performance, the performance of the conventional phrase-based system without using any neural network-based component.

RNN E NCODER –D ECODER · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_5dc149f4
Hence, we propose to use a bidirectional RNN (BiRNN, Schuster and Paliwal, 1997), which has been successfully used recently in speech recognition (see, e.g., Graves et al., 2013).

E NCODER : B IDIRECTIONAL RNN FOR ANNOTATING SEQUENCES · page 4 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_67dd9d1a
It is clear from the table that in all the cases, the proposed RNNsearch outperforms the conventional RNNencdec.

Q UANTITATIVE RESULTS · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_530a221a
This is a significant achievement, considering that Moses uses a separate monolingual corpus (418M words) in addition to the parallel corpora we used to train the RNNsearch and RNNencdec.

Q UANTITATIVE RESULTS · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_62c0a117
On the other hand, both RNNsearch-30 and RNNsearch-50 are more robust to the length of the sentences.

Q UANTITATIVE RESULTS · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_dfffa427
This superiority of the proposed model over the basic encoder–decoder is further confirmed by the fact that the RNNsearch-30 even outperforms RNNencdec-50 (see Table 1).

Q UANTITATIVE RESULTS · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e1f2a17b
Note that RNNsearch-50⋆ was trained much longer until the performance on the development set stopped improving.

Q UANTITATIVE RESULTS · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_6a5348e0
In this paper, we proposed a novel architecture that addresses this issue.

C ONCLUSION · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_708fc3ce
The experiment revealed that the proposed RNNsearch outperforms the conventional encoder–decoder model (RNNencdec) significantly, regardless of the sentence length and that it is much more robust to the length of … (excerpt)

C ONCLUSION · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_035ea1ab
This will be required for the model to be more widely used and to match the performance of current state-of-the-art machine translation systems in all contexts.

C ONCLUSION · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1409.0473) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.