  We introduce HarperValleyBank, a free, public domain spoken dialog corpus.
The data simulate simple consumer banking interactions, containing about 23
hours of audio from 1,446 human-human conversations between 59 unique speakers.
We selected intents and utterance templates to allow realistic variation while
controlling overall task complexity and limiting vocabulary size to about 700
unique words. We provide audio data along with transcripts and annotations for
speaker identity, caller intent, dialog actions, and emotional valence. The
data size and domain specificity makes for quick transcription experiments with
modern end-to-end neural approaches. Further, we provide baselines for
representation learning, adapting recent work to embed waveforms for downstream
prediction tasks. Our experiments show that tasks using our annotations are
sensitive to both the model choice and corpus size.
