  Ride-hailing services, such as Didi Chuxing, Lyft, and Uber, arrange
thousands of cars to meet ride requests throughout the day. We consider a
Markov decision process (MDP) model of a ride-hailing service system, framing
it as a reinforcement learning (RL) problem. The simultaneous control of many
agents (cars) presents a challenge for the MDP optimization because the action
space grows exponentially with the number of cars. We propose a special
decomposition for the MDP actions by sequentially assigning tasks to the
drivers. The new actions structure resolves the scalability problem and enables
the use of deep RL algorithms for control policy optimization. We demonstrate
the benefit of our proposed decomposition with a numerical experiment based on
real data from Didi Chuxing.
