  Recent multilingual pretrained language models (mPLMs) have been shown to
encode strong language-specific signals, which are not explicitly provided
during pretraining. It remains an open question whether it is feasible to
employ mPLMs to measure language similarity, and subsequently use the
similarity results to select source languages for boosting cross-lingual
transfer. To investigate this, we propose mPLMSim, a language similarity
measure that induces the similarities across languages from mPLMs using
multi-parallel corpora. Our study shows that mPLM-Sim exhibits moderately high
correlations with linguistic similarity measures, such as lexicostatistics,
genealogical language family, and geographical sprachbund. We also conduct a
case study on languages with low correlation and observe that mPLM-Sim yields
more accurate similarity results. Additionally, we find that similarity results
vary across different mPLMs and different layers within an mPLM. We further
investigate whether mPLMSim is effective for zero-shot cross-lingual transfer
by conducting experiments on both low-level syntactic tasks and high-level
semantic tasks. The experimental results demonstrate that mPLM-Sim is capable
of selecting better source languages than linguistic measures, resulting in a
1%-2% improvement in zero-shot cross-lingual transfer performance.
