  Neural language models, which reach state-of-the-art results on most natural
language processing tasks, are trained on large text corpora that inevitably
contain value-burdened content and often capture undesirable biases, which the
models reflect. This case study focuses on the political biases of pre-trained
encoders in Czech and compares them with a representative value survey. Because
Czech is a gendered language, we also measure how the grammatical gender
coincides with responses to men and women in the survey. We introduce a novel
method for measuring the model's perceived political values. We find that the
models do not assign statement probability following value-driven reasoning,
and there is no systematic difference between feminine and masculine sentences.
We conclude that BERT-sized models do not manifest systematic alignment with
political values and that the biases observed in the models are rather due to
superficial imitation of training data patterns than systematic value beliefs
encoded in the models.
