  Recent advances in Neural Machine Translation (NMT) show that adding
syntactic information to NMT systems can improve the quality of their
translations. Most existing work utilizes some specific types of
linguistically-inspired tree structures, like constituency and dependency parse
trees. This is often done via a standard RNN decoder that operates on a
linearized target tree structure. However, it is an open question of what
specific linguistic formalism, if any, is the best structural representation
for NMT. In this paper, we (1) propose an NMT model that can naturally generate
the topology of an arbitrary tree structure on the target side, and (2)
experiment with various target tree structures. Our experiments show the
surprising result that our model delivers the best improvements with balanced
binary trees constructed without any linguistic knowledge; this model
outperforms standard seq2seq models by up to 2.1 BLEU points, and other methods
for incorporating target-side syntax by up to 0.7 BLEU.
