  A fundamental question in any peer-to-peer ridesharing system is how to, both
effectively and efficiently, dispatch user's ride requests to the right driver
in real time. Traditional rule-based solutions usually work on a simplified
problem setting, which requires a sophisticated hand-crafted weight design for
either centralized authority control or decentralized multi-agent scheduling
systems. Although recent approaches have used reinforcement learning to provide
centralized combinatorial optimization algorithms with informative weight
values, their single-agent setting can hardly model the complex interactions
between drivers and orders. In this paper, we address the order dispatching
problem using multi-agent reinforcement learning (MARL), which follows the
distributed nature of the peer-to-peer ridesharing problem and possesses the
ability to capture the stochastic demand-supply dynamics in large-scale
ridesharing scenarios. Being more reliable than centralized approaches, our
proposed MARL solutions could also support fully distributed execution through
recent advances in the Internet of Vehicles (IoV) and the Vehicle-to-Network
(V2N). Furthermore, we adopt the mean field approximation to simplify the local
interactions by taking an average action among neighborhoods. The mean field
approximation is capable of globally capturing dynamic demand-supply variations
by propagating many local interactions between agents and the environment. Our
extensive experiments have shown the significant improvements of MARL order
dispatching algorithms over several strong baselines on the gross merchandise
volume (GMV), and order response rate measures. Besides, the simulated
experiments with real data have also justified that our solution can alleviate
the supply-demand gap during the rush hours, thus possessing the capability of
reducing traffic congestion.
