  Achieving the capability of adapting to ever-changing environments is a
critical step towards building fully autonomous robots that operate safely in
complicated scenarios. In multiagent competitive scenarios, agents may have to
adapt to new opponents with previously unseen behaviors by learning from the
interaction experiences between the ego-agent and the opponent. However, this
adaptation is susceptible to opponent exploitation. As the ego-agent updates
its own behavior to exploit the opponent, its own behavior could become more
exploitable as a result of overfitting to this specific opponent's behavior. To
overcome this difficulty, we developed a safe adaptation approach in which the
ego-agent is trained against a regularized opponent model, which effectively
avoids overfitting and consequently improves the robustness of the ego-agent's
policy. We evaluated our approach in the Mujoco domain with two competing
agents. The experiment results suggest that our approach effectively achieves
both adaptation to the specific opponent that the ego-agent is interacting with
and maintaining low exploitability to other possible opponent exploitation.
