  Neural Machine Translation (NMT) performs poor on the low-resource language
pair $(X,Z)$, especially when $Z$ is a rare language. By introducing another
rich language $Y$, we propose a novel triangular training architecture (TA-NMT)
to leverage bilingual data $(Y,Z)$ (may be small) and $(X,Y)$ (can be rich) to
improve the translation performance of low-resource pairs. In this triangular
architecture, $Z$ is taken as the intermediate latent variable, and translation
models of $Z$ are jointly optimized with a unified bidirectional EM algorithm
under the goal of maximizing the translation likelihood of $(X,Y)$. Empirical
results demonstrate that our method significantly improves the translation
quality of rare languages on MultiUN and IWSLT2012 datasets, and achieves even
better performance combining back-translation methods.
