  Neural Machine Translation (NMT) can be used to generate fluent output. As
such, language models have been investigated for incorporation with NMT. In
prior investigations, two models have been used: a translation model and a
language model. The translation model's predictions are weighted by the
language model with a hand-crafted ratio in advance. However, these approaches
fail to adopt the language model weighting with regard to the translation
history. In another line of approach, language model prediction is incorporated
into the translation model by jointly considering source and target
information. However, this line of approach is limited because it largely
ignores the adequacy of the translation output.
  Accordingly, this work employs two mechanisms, the translation model and the
language model, with an attentive architecture to the language model as an
auxiliary element of the translation model. Compared with previous work in
English--Japanese machine translation using a language model, the experimental
results obtained with the proposed Dynamic Fusion mechanism improve BLEU and
Rank-based Intuitive Bilingual Evaluation Scores (RIBES) scores. Additionally,
in the analyses of the attention and predictivity of the language model, the
Dynamic Fusion mechanism allows predictive language modeling that conforms to
the appropriate grammatical structure.
