  The complete sharing of parameters for multilingual translation (1-1) has
been the mainstream approach in current research. However, degraded performance
due to the capacity bottleneck and low maintainability hinders its extensive
adoption in industries. In this study, we revisit the multilingual neural
machine translation model that only share modules among the same languages (M2)
as a practical alternative to 1-1 to satisfy industrial requirements. Through
comprehensive experiments, we identify the benefits of multi-way training and
demonstrate that the M2 can enjoy these benefits without suffering from the
capacity bottleneck. Furthermore, the interlingual space of the M2 allows
convenient modification of the model. By leveraging trained modules, we find
that incrementally added modules exhibit better performance than singly trained
models. The zero-shot performance of the added modules is even comparable to
supervised models. Our findings suggest that the M2 can be a competent
candidate for multilingual translation in industries.
