  This study probes the vulnerabilities of cooperative multi-agent
reinforcement learning (c-MARL) under adversarial attacks, a critical
determinant of c-MARL's worst-case performance prior to real-world
implementation. Current observation-based attacks, constrained by white-box
assumptions, overlook c-MARL's complex multi-agent interactions and cooperative
objectives, resulting in impractical and limited attack capabilities. To
address these shortcomes, we propose Adversarial Minority Influence (AMI), a
practical and strong for c-MARL. AMI is a practical black-box attack and can be
launched without knowing victim parameters. AMI is also strong by considering
the complex multi-agent interaction and the cooperative goal of agents,
enabling a single adversarial agent to unilaterally misleads majority victims
to form targeted worst-case cooperation. This mirrors minority influence
phenomena in social psychology. To achieve maximum deviation in victim policies
under complex agent-wise interactions, our unilateral attack aims to
characterize and maximize the impact of the adversary on the victims. This is
achieved by adapting a unilateral agent-wise relation metric derived from
mutual information, thereby mitigating the adverse effects of victim influence
on the adversary. To lead the victims into a jointly detrimental scenario, our
targeted attack deceives victims into a long-term, cooperatively harmful
situation by guiding each victim towards a specific target, determined through
a trial-and-error process executed by a reinforcement learning agent. Through
AMI, we achieve the first successful attack against real-world robot swarms and
effectively fool agents in simulated environments into collectively worst-case
scenarios, including Starcraft II and Multi-agent Mujoco. The source code and
demonstrations can be found at: https://github.com/DIG-Beihang/AMI.
