  Massively multilingual neural machine translation (MMNMT) has been proven to
enhance the translation quality of low-resource languages. In this paper, we
empirically investigate the translation robustness of Indonesian-Chinese
translation in the face of various naturally occurring noise. To assess this,
we create a robustness evaluation benchmark dataset for Indonesian-Chinese
translation. This dataset is automatically translated into Chinese using four
NLLB-200 models of different sizes. We conduct both automatic and human
evaluations. Our in-depth analysis reveal the correlations between translation
error types and the types of noise present, how these correlations change
across different model sizes, and the relationships between automatic
evaluation indicators and human evaluation indicators. The dataset is publicly
available at https://github.com/tjunlp-lab/ID-ZH-MTRobustEval.
