  The multi-agent reinforcement learning systems (MARL) based on the Markov
decision process (MDP) have emerged in many critical applications. To improve
the robustness/defense of MARL systems against adversarial attacks, the study
of various adversarial attacks on reinforcement learning systems is very
important. Previous works on adversarial attacks considered some possible
features to attack in MDP, such as the action poisoning attacks, the reward
poisoning attacks, and the state perception attacks. In this paper, we propose
a brand-new form of attack called the camouflage attack in the MARL systems. In
the camouflage attack, the attackers change the appearances of some objects
without changing the actual objects themselves; and the camouflaged appearances
may look the same to all the targeted recipient (victim) agents. The
camouflaged appearances can mislead the recipient agents to misguided actions.
We design algorithms that give the optimal camouflage attacks minimizing the
rewards of recipient agents. Our numerical and theoretical results show that
camouflage attacks can rival the more conventional, but likely more difficult
state perception attacks. We also investigate cost-constrained camouflage
attacks and showed numerically how cost budgets affect the attack performance.
