  We show that unsupervised sequence-segmentation performance can be
transferred to extremely low-resource languages by pre-training a Masked
Segmental Language Model (Downey et al., 2021) multilingually. Further, we show
that this transfer can be achieved by training over a collection of
low-resource languages that are typologically similar (but phylogenetically
unrelated) to the target language. In our experiments, we transfer from a
collection of 10 Indigenous American languages (AmericasNLP, Mager et al.,
2021) to K'iche', a Mayan language. We compare our multilingual model to a
monolingual (from-scratch) baseline, as well as a model pre-trained on Quechua
only. We show that the multilingual pre-trained approach yields consistent
segmentation quality across target dataset sizes, exceeding the monolingual
baseline in 6/10 experimental settings. Our model yields especially strong
results at small target sizes, including a zero-shot performance of 20.6 F1.
These results have promising implications for low-resource NLP pipelines
involving human-like linguistic units, such as the sparse transcription
framework proposed by Bird (2020).
