  Approximating Martingale Process (AMP) is proven to be effective for variance
reduction in reinforcement learning (RL) in specific cases such as Multiclass
Queueing Networks. However, in the already proven cases, the state space is
relatively small and all possible state transitions can be iterated through. In
this paper, we consider systems in which state space is large and have
uncertainties when considering state transitions, thus making AMP a generalized
variance-reduction method in RL. Specifically, we will investigate the
application of AMP in ride-hailing systems like Uber, where Proximal Policy
Optimization (PPO) is incorporated to optimize the policy of matching drivers
and customers.
