  We probe the layers in multilingual BERT (mBERT) for phylogenetic and
geographic language signals across 100 languages and compute language distances
based on the mBERT representations. We 1) employ the language distances to
infer and evaluate language trees, finding that they are close to the reference
family tree in terms of quartet tree distance, 2) perform distance matrix
regression analysis, finding that the language distances can be best explained
by phylogenetic and worst by structural factors and 3) present a novel measure
for measuring diachronic meaning stability (based on cross-lingual
representation variability) which correlates significantly with published
ranked lists based on linguistic approaches. Our results contribute to the
nascent field of typological interpretability of cross-lingual text
representations.
