  We investigate the compositional structure of message vectors computed by a
deep network trained on a communication game. By comparing truth-conditional
representations of encoder-produced message vectors to human-produced referring
expressions, we are able to identify aligned (vector, utterance) pairs with the
same meaning. We then search for structured relationships among these aligned
pairs to discover simple vector space transformations corresponding to
negation, conjunction, and disjunction. Our results suggest that neural
representations are capable of spontaneously developing a "syntax" with
functional analogues to qualitative properties of natural language.
