  This work extends an existing virtual multi-agent platform called RoboSumo to
create TripleSumo -- a platform for investigating multi-agent cooperative
behaviors in continuous action spaces, with physical contact in an adversarial
environment. In this paper we investigate a scenario in which two agents,
namely `Bug' and `Ant', must team up and push another agent `Spider' out of the
arena. To tackle this goal, the newly added agent `Bug' is trained during an
ongoing match between `Ant' and `Spider'. `Bug' must develop awareness of the
other agents' actions, infer the strategy of both sides, and eventually learn
an action policy to cooperate. The reinforcement learning algorithm Deep
Deterministic Policy Gradient (DDPG) is implemented with a hybrid reward
structure combining dense and sparse rewards. The cooperative behavior is
quantitatively evaluated by the mean probability of winning the match and mean
number of steps needed to win.
