  Grounded language acquisition -- learning how language-based interactions
refer to the world around them -- is amajor area of research in robotics, NLP,
and HCI. In practice the data used for learning consists almost entirely of
textual descriptions, which tend to be cleaner, clearer, and more grammatical
than actual human interactions. In this work, we present the Grounded Language
Dataset (GoLD), a multimodal dataset of common household objects described by
people using either spoken or written language. We analyze the differences and
present an experiment showing how the different modalities affect language
learning from human in-put. This will enable researchers studying the
intersection of robotics, NLP, and HCI to better investigate how the multiple
modalities of image, text, and speech interact, as well as show differences in
the vernacular of these modalities impact results.
