  Neural models of dialog rely on generalized latent representations of
language. This paper introduces a novel training procedure which explicitly
learns multiple representations of language at several levels of granularity.
The multi-granularity training algorithm modifies the mechanism by which
negative candidate responses are sampled in order to control the granularity of
learned latent representations. Strong performance gains are observed on the
next utterance retrieval task using both the MultiWOZ dataset and the Ubuntu
dialog corpus. Analysis significantly demonstrates that multiple granularities
of representation are being learned, and that multi-granularity training
facilitates better transfer to downstream tasks.
