  Automated algorithm configuration relieves users from tedious,
trial-and-error tuning tasks. A popular algorithm configuration tuning paradigm
is dynamic algorithm configuration (DAC), in which an agent learns dynamic
configuration policies across instances by reinforcement learning (RL).
However, in many complex algorithms, there may exist different types of
configuration hyperparameters, and such heterogeneity may bring difficulties
for classic DAC which uses a single-agent RL policy. In this paper, we aim to
address this issue and propose multi-agent DAC (MA-DAC), with one agent working
for one type of configuration hyperparameter. MA-DAC formulates the dynamic
configuration of a complex algorithm with multiple types of hyperparameters as
a contextual multi-agent Markov decision process and solves it by a cooperative
multi-agent RL (MARL) algorithm. To instantiate, we apply MA-DAC to a
well-known optimization algorithm for multi-objective optimization problems.
Experimental results show the effectiveness of MA-DAC in not only achieving
superior performance compared with other configuration tuning approaches based
on heuristic rules, multi-armed bandits, and single-agent RL, but also being
capable of generalizing to different problem classes. Furthermore, we release
the environments in this paper as a benchmark for testing MARL algorithms, with
the hope of facilitating the application of MARL.
