  Multi-agent reinforcement learning offers a way to study how communication
could emerge in communities of agents needing to solve specific problems. In
this paper, we study the emergence of communication in the negotiation
environment, a semi-cooperative model of agent interaction. We introduce two
communication protocols -- one grounded in the semantics of the game, and one
which is \textit{a priori} ungrounded and is a form of cheap talk. We show that
self-interested agents can use the pre-grounded communication channel to
negotiate fairly, but are unable to effectively use the ungrounded channel.
However, prosocial agents do learn to use cheap talk to find an optimal
negotiating strategy, suggesting that cooperation is necessary for language to
emerge. We also study communication behaviour in a setting where one agent
interacts with agents in a community with different levels of prosociality and
show how agent identifiability can aid negotiation.
