  Many real-world applications involve teams of agents that have to coordinate
their actions to reach a common goal against potential adversaries. This paper
focuses on zero-sum games where a team of players faces an opponent, as is the
case, for example, in Bridge, collusion in poker, and collusion in bidding. The
possibility for the team members to communicate before gameplay---that is,
coordinate their strategies ex ante---makes the use of behavioral strategies
unsatisfactory. We introduce Soft Team Actor-Critic (STAC) as a solution to the
team's coordination problem that does not require any prior domain knowledge.
STAC allows team members to effectively exploit ex ante communication via
exogenous signals that are shared among the team. STAC reaches near-optimal
coordinated strategies both in perfectly observable and partially observable
games, where previous deep RL algorithms fail to reach optimal coordinated
behaviors.
