  The objective of this article is to optimize the overall traffic flow on
freeways using multiple ramp metering controls plus its complementary Dynamic
Speed Limits (DSLs). An optimal freeway operation can be reached when
minimizing the difference between the freeway density and the critical ratio
for maximum traffic flow. In this article, a Multi-Agent Reinforcement Learning
for Freeways Control (MARL-FWC) system for ramps metering and DSLs is proposed.
MARL-FWC introduces a new microscopic framework at the network level based on
collaborative Markov Decision Process modeling (Markov game) and an associated
cooperative Q-learning algorithm. The technique incorporates payoff propagation
(Max-Plus algorithm) under the coordination graphs framework, particularly
suited for optimal control purposes. MARL-FWC provides three control designs:
fully independent, fully distributed, and centralized; suited for different
network architectures. MARL-FWC was extensively tested in order to assess the
proposed model of the joint payoff, as well as the global payoff. Experiments
are conducted with heavy traffic flow under the renowned VISSIM traffic
simulator to evaluate MARL-FWC. The experimental results show a significant
decrease in the total travel time and an increase in the average speed (when
compared with the base case) while maintaining an optimal traffic flow.
