  The ability to pick up on language signals in an ongoing interaction is
crucial for future machine learning models to collaborate and interact with
humans naturally. In this paper, we present an initial study that evaluates
intra-episodic feedback given in a collaborative setting. We use a referential
language game as a controllable example of a task-oriented collaborative joint
activity. A teacher utters a referring expression generated by a well-known
symbolic algorithm (the "Incremental Algorithm") as an initial instruction and
then monitors the follower's actions to possibly intervene with intra-episodic
feedback (which does not explicitly have to be requested). We frame this task
as a reinforcement learning problem with sparse rewards and learn a follower
policy for a heuristic teacher. Our results show that intra-episodic feedback
allows the follower to generalize on aspects of scene complexity and performs
better than providing only the initial statement.
