  In this work, we report on the effectiveness of our efforts to tailor the
personality and conversational style of a conversational agent based on GPT-3.5
and GPT-4 through prompts. We use three personality dimensions with two levels
each to create eight conversational agents archetypes. Ten conversations were
collected per chatbot, of ten exchanges each, generating 1600 exchanges across
GPT-3.5 and GPT-4. Using Linguistic Inquiry and Word Count (LIWC) analysis, we
compared the eight agents on language elements including clout, authenticity,
and emotion. Four language cues were significantly distinguishing in GPT-3.5,
while twelve were distinguishing in GPT-4. With thirteen out of a total
nineteen cues in LIWC appearing as significantly distinguishing, our results
suggest possible novel prompting approaches may be needed to better suit the
creation and evaluation of persistent conversational agent personalities or
language styles.
