  Transformer models achieve remarkable success in Neural Machine Translation.
Many efforts have been devoted to deepening the Transformer by stacking several
units (i.e., a combination of Multihead Attentions and FFN) in a cascade, while
the investigation over multiple parallel units draws little attention. In this
paper, we propose the Multi-Unit Transformers (MUTE), which aim to promote the
expressiveness of the Transformer by introducing diverse and complementary
units. Specifically, we use several parallel units and show that modeling with
multiple units improves model performance and introduces diversity. Further, to
better leverage the advantage of the multi-unit setting, we design biased
module and sequential dependency that guide and encourage complementariness
among different units. Experimental results on three machine translation tasks,
the NIST Chinese-to-English, WMT'14 English-to-German and WMT'18
Chinese-to-English, show that the MUTE models significantly outperform the
Transformer-Base, by up to +1.52, +1.90 and +1.10 BLEU points, with only a mild
drop in inference speed (about 3.1%). In addition, our methods also surpass the
Transformer-Big model, with only 54\% of its parameters. These results
demonstrate the effectiveness of the MUTE, as well as its efficiency in both
the inference process and parameter usage.
