  Acquiring your first language is an incredible feat and not easily
duplicated. Learning to communicate using nothing but a few pictureless books,
a corpus, would likely be impossible even for humans. Nevertheless, this is the
dominating approach in most natural language processing today. As an
alternative, we propose the use of situated interactions between agents as a
driving force for communication, and the framework of Deep Recurrent Q-Networks
for evolving a shared language grounded in the provided environment. We task
the agents with interactive image search in the form of the game Guess Who?.
The images from the game provide a non trivial environment for the agents to
discuss and a natural grounding for the concepts they decide to encode in their
communication. Our experiments show that the agents learn not only to encode
physical concepts in their words, i.e. grounding, but also that the agents
learn to hold a multi-step dialogue remembering the state of the dialogue from
step to step.
