  Language embeds information about social, cultural, and political values
people hold. Prior work has explored social and potentially harmful biases
encoded in Pre-Trained Language models (PTLMs). However, there has been no
systematic study investigating how values embedded in these models vary across
cultures. In this paper, we introduce probes to study which values across
cultures are embedded in these models, and whether they align with existing
theories and cross-cultural value surveys. We find that PTLMs capture
differences in values across cultures, but those only weakly align with
established value surveys. We discuss implications of using mis-aligned models
in cross-cultural settings, as well as ways of aligning PTLMs with value
surveys.
