  In this paper, we present a Bayesian multilingual document model for learning
language-independent document embeddings. The model is an extension of BaySMM
[Kesiraju et al 2020] to the multilingual scenario. It learns to represent the
document embeddings in the form of Gaussian distributions, thereby encoding the
uncertainty in its covariance. We propagate the learned uncertainties through
linear classifiers that benefit zero-shot cross-lingual topic identification.
Our experiments on 17 languages show that the proposed multilingual Bayesian
document model performs competitively, when compared to other systems based on
large-scale neural networks (LASER, XLM-R, mUSE) on 8 high-resource languages,
and outperforms these systems on 9 mid-resource languages. We revisit
cross-lingual topic identification in zero-shot settings by taking a deeper
dive into current datasets, baseline systems and the languages covered. We
identify shortcomings in the existing evaluation protocol (MLDoc dataset), and
propose a robust alternative scheme, while also extending the cross-lingual
experimental setup to 17 languages. Finally, we consolidate the observations
from all our experiments, and discuss points that can potentially benefit the
future research works in applications relying on cross-lingual transfers.
