  In this work we analyze Multi-Agent Advantage Actor-Critic (MA2C) a recently
proposed multi-agent reinforcement learning algorithm that can be applied to
adaptive traffic signal control (ATSC) problems. To evaluate its potential we
compare MA2C with Independent Advantage Actor-Critic (IA2C) and other
Reinforcement Learning or heuristic based algorithms. Specifically, we analyze
MA2C theoretically with the framework provided by non-Markov decision
processes, which allows a deeper insight of the algorithm, and we critically
examine the effectiveness and the robustness of the method by testing it in two
traffic areas located in Bologna (Italy) simulated in SUMO, a software modeling
tool for ATSC problems. Our results indicate that MA2C, trained with
pseudo-random vehicle flows, is a promising technique able to outperform the
alternative methods.
