  While neural machine translation (NMT) has achieved state-of-the-art
translation performance, it is unable to capture the alignment between the
input and output during the translation process. The lack of alignment in NMT
models leads to three problems: it is hard to (1) interpret the translation
process, (2) impose lexical constraints, and (3) impose structural constraints.
To alleviate these problems, we propose to introduce explicit phrase alignment
into the translation process of arbitrary NMT models. The key idea is to build
a search space similar to that of phrase-based statistical machine translation
for NMT where phrase alignment is readily available. We design a new decoding
algorithm that can easily impose lexical and structural constraints.
Experiments show that our approach makes the translation process of NMT more
interpretable without sacrificing translation quality. In addition, our
approach achieves significant improvements in lexically and structurally
constrained translation tasks.
