  Recent research has extended beyond assessing the performance of Large
Language Models (LLMs) to examining their characteristics from a psychological
standpoint, acknowledging the necessity of understanding their behavioral
characteristics. The administration of personality tests to LLMs has emerged as
a noteworthy area in this context. However, the suitability of employing
psychological scales, initially devised for humans, on LLMs is a matter of
ongoing debate. Our study aims to determine the reliability of applying
personality assessments to LLMs, explicitly investigating whether LLMs
demonstrate consistent personality traits. Analyzing responses under 2,500
settings reveals that gpt-3.5-turbo shows consistency in responses to the Big
Five Inventory, indicating a high degree of reliability. Furthermore, our
research explores the potential of gpt-3.5-turbo to emulate diverse
personalities and represent various groups, which is a capability increasingly
sought after in social sciences for substituting human participants with LLMs
to reduce costs. Our findings reveal that LLMs have the potential to represent
different personalities with specific prompt instructions. By shedding light on
the personalization of LLMs, our study endeavors to pave the way for future
explorations in this field. We have made our experimental results and the
corresponding code openly accessible via
https://github.com/CUHK-ARISE/LLMPersonality.
