  Recent work on multilingual neural machine translation reported competitive
performance with respect to bilingual models and surprisingly good performance
even on (zeroshot) translation directions not observed at training time. We
investigate here a zero-shot translation in a particularly lowresource
multilingual setting. We propose a simple iterative training procedure that
leverages a duality of translations directly generated by the system for the
zero-shot directions. The translations produced by the system (sub-optimal
since they contain mixed language from the shared vocabulary), are then used
together with the original parallel data to feed and iteratively re-train the
multilingual network. Over time, this allows the system to learn from its own
generated and increasingly better output. Our approach shows to be effective in
improving the two zero-shot directions of our multilingual model. In
particular, we observed gains of about 9 BLEU points over a baseline
multilingual model and up to 2.08 BLEU over a pivoting mechanism using two
bilingual models. Further analysis shows that there is also a slight
improvement in the non-zero-shot language directions.
