  We describe a neural transducer that maintains the flexibility of standard
sequence-to-sequence (seq2seq) models while incorporating hierarchical phrases
as a source of inductive bias during training and as explicit constraints
during inference. Our approach trains two models: a discriminative parser based
on a bracketing transduction grammar whose derivation tree hierarchically
aligns source and target phrases, and a neural seq2seq model that learns to
translate the aligned phrases one-by-one. We use the same seq2seq model to
translate at all phrase scales, which results in two inference modes: one mode
in which the parser is discarded and only the seq2seq component is used at the
sequence-level, and another in which the parser is combined with the seq2seq
model. Decoding in the latter mode is done with the cube-pruned CKY algorithm,
which is more involved but can make use of new translation rules during
inference. We formalize our model as a source-conditioned synchronous grammar
and develop an efficient variational inference algorithm for training. When
applied on top of both randomly initialized and pretrained seq2seq models, we
find that both inference modes performs well compared to baselines on small
scale machine translation benchmarks.
