  Backdoor attacks on reinforcement learning implant a backdoor in a victim
agent's policy. Once the victim observes the trigger signal, it will switch to
the abnormal mode and fail its task. Most of the attacks assume the adversary
can arbitrarily modify the victim's observations, which may not be practical.
One work proposes to let one adversary agent use its actions to affect its
opponent in two-agent competitive games, so that the opponent quickly fails
after observing certain trigger actions. However, in multiagent collaborative
systems, agents may not always be able to observe others. When and how much the
adversary agent can affect others are uncertain, and we want the adversary
agent to trigger others for as few times as possible. To solve this problem, we
first design a novel training framework to produce auxiliary rewards that
measure the extent to which the other agents'observations being affected. Then
we use the auxiliary rewards to train a trigger policy which enables the
adversary agent to efficiently affect the others' observations. Given these
affected observations, we further train the other agents to perform abnormally.
Extensive experiments demonstrate that the proposed method enables the
adversary agent to lure the others into the abnormal mode with only a few
actions.
