  Do large language models (LLMs) have their own worldviews and personality
tendencies? Simulations in which an LLM was asked to answer subjective
questions were conducted more than 1 million times. Comparison of the responses
from different LLMs with real data from the European Social Survey (ESS)
suggests that the effect of prompts on bias and variability is fundamental,
highlighting major cultural, age, and gender biases. Methods for measuring the
difference between LLMs and survey data are discussed, such as calculating
weighted means and a new proposed measure inspired by Jaccard similarity. We
conclude that it is important to analyze the robustness and variability of
prompts before using LLMs to model individual decisions or collective behavior,
as their imitation abilities are approximate at best.
