  We introduce and demonstrate how to effectively train multilingual machine
translation models with pixel representations. We experiment with two different
data settings with a variety of language and script coverage, demonstrating
improved performance compared to subword embeddings. We explore various
properties of pixel representations such as parameter sharing within and across
scripts to better understand where they lead to positive transfer. We observe
that these properties not only enable seamless cross-lingual transfer to unseen
scripts, but make pixel representations more data-efficient than alternatives
such as vocabulary expansion. We hope this work contributes to more extensible
multilingual models for all languages and scripts.
