  Long-term fairness is an important factor of consideration in designing and
deploying learning-based decision systems in high-stake decision-making
contexts. Recent work has proposed the use of Markov Decision Processes (MDPs)
to formulate decision-making with long-term fairness requirements in
dynamically changing environments, and demonstrated major challenges in
directly deploying heuristic and rule-based policies that worked well in static
environments. We show that policy optimization methods from deep reinforcement
learning can be used to find strictly better decision policies that can often
achieve both higher overall utility and less violation of the fairness
requirements, compared to previously-known strategies. In particular, we
propose new methods for imposing fairness requirements in policy optimization
by regularizing the advantage evaluation of different actions. Our proposed
methods make it easy to impose fairness constraints without reward engineering
or sacrificing training efficiency. We perform detailed analyses in three
established case studies, including attention allocation in incident
monitoring, bank loan approval, and vaccine distribution in population
networks.
