  Reinforcement learning (RL) is inspired by the way human infants and animals
learn from the environment. The setting is somewhat idealized because, in
actual tasks, other agents in the environment have their own goals and behave
adaptively to the ego agent. To thrive in those environments, the agent needs
to influence other agents so their actions become more helpful and less
harmful. Research in computational economics distills two ways to influence
others directly: by providing tangible goods (mechanism design) and by
providing information (information design). This work investigates information
design problems for a group of RL agents. The main challenges are two-fold. One
is the information provided will immediately affect the transition of the agent
trajectories, which introduces additional non-stationarity. The other is the
information can be ignored, so the sender must provide information that the
receiver is willing to respect. We formulate the Markov signaling game, and
develop the notions of signaling gradient and the extended obedience
constraints that address these challenges. Our algorithm is efficient on
various mixed-motive tasks and provides further insights into computational
economics. Our code is publicly available at
https://github.com/YueLin301/InformationDesignMARL.
