  We create and release the first publicly available commercial customer
service corpus with annotated relational segments. Human-computer data from
three live customer service Intelligent Virtual Agents (IVAs) in the domains of
travel and telecommunications were collected, and reviewers marked all text
that was deemed unnecessary to the determination of user intention. After
merging the selections of multiple reviewers to create highlighted texts, a
second round of annotation was done to determine the classes of language
present in the highlighted sections such as the presence of Greetings,
Backstory, Justification, Gratitude, Rants, or Emotions. This resulting corpus
is a valuable resource for improving the quality and relational abilities of
IVAs. As well as discussing the corpus itself, we compare the usage of such
language in human-human interactions on TripAdvisor forums. We show that
removal of this language from task-based inputs has a positive effect on IVA
understanding by both an increase in confidence and improvement in responses,
demonstrating the need for automated methods of its discovery.
