  When autonomous agents interact in the same environment, they must often
cooperate to achieve their goals. One way for agents to cooperate effectively
is to form a team, make a binding agreement on a joint plan, and execute it.
However, when agents are self-interested, the gains from team formation must be
allocated appropriately to incentivize agreement. Various approaches for
multi-agent negotiation have been proposed, but typically only work for
particular negotiation protocols. More general methods usually require human
input or domain-specific data, and so do not scale. To address this, we propose
a framework for training agents to negotiate and form teams using deep
reinforcement learning. Importantly, our method makes no assumptions about the
specific negotiation protocol, and is instead completely experience driven. We
evaluate our approach on both non-spatial and spatially extended team-formation
negotiation environments, demonstrating that our agents beat hand-crafted bots
and reach negotiation outcomes consistent with fair solutions predicted by
cooperative game theory. Additionally, we investigate how the physical location
of agents influences negotiation outcomes.
