  There has been concern about ideological basis and possible discrimination in
text generated by Large Language Models (LLMs). We test possible value biases
in ChatGPT using a psychological value theory. We designed a simple experiment
in which we used a number of different probes derived from the Schwartz basic
value theory (items from the revised Portrait Value Questionnaire, the value
type definitions, value names). We prompted ChatGPT via the OpenAI API
repeatedly to generate text and then analyzed the generated corpus for value
content with a theory-driven value dictionary using a bag of words approach.
Overall, we found little evidence of explicit value bias. The results showed
sufficient construct and discriminant validity for the generated text in line
with the theoretical predictions of the psychological model, which suggests
that the value content was carried through into the outputs with high fidelity.
We saw some merging of socially oriented values, which may suggest that these
values are less clearly differentiated at a linguistic level or alternatively,
this mixing may reflect underlying universal human motivations. We outline some
possible applications of our findings for both applications of ChatGPT for
corporate usage and policy making as well as future research avenues. We also
highlight possible implications of this relatively high-fidelity replication of
motivational content using a linguistic model for the theorizing about human
values.
