  Despite the impressive quality improvements yielded by neural machine
translation (NMT) systems, controlling their translation output to adhere to
user-provided terminology constraints remains an open problem. We describe our
approach to constrained neural decoding based on finite-state machines and
multi-stack decoding which supports target-side constraints as well as
constraints with corresponding aligned input text spans. We demonstrate the
performance of our framework on multiple translation tasks and motivate the
need for constrained decoding with attentions as a means of reducing
misplacement and duplication when translating user constraints.
