  Conversational tones -- the manners and attitudes in which speakers
communicate -- are essential to effective communication. Amidst the increasing
popularization of Large Language Models (LLMs) over recent years, it becomes
necessary to characterize the divergences in their conversational tones
relative to humans. However, existing investigations of conversational
modalities rely on pre-existing taxonomies or text corpora, which suffer from
experimenter bias and may not be representative of real-world distributions for
the studies' psycholinguistic domains. Inspired by methods from cognitive
science, we propose an iterative method for simultaneously eliciting
conversational tones and sentences, where participants alternate between two
tasks: (1) one participant identifies the tone of a given sentence and (2) a
different participant generates a sentence based on that tone. We run 100
iterations of this process with human participants and GPT-4, then obtain a
dataset of sentences and frequent conversational tones. In an additional
experiment, humans and GPT-4 annotated all sentences with all tones. With data
from 1,339 human participants, 33,370 human judgments, and 29,900 GPT-4
queries, we show how our approach can be used to create an interpretable
geometric representation of relations between conversational tones in humans
and GPT-4. This work demonstrates how combining ideas from machine learning and
cognitive science can address challenges in human-computer interactions.
