  Robust Markov Decision Processes (RMDPs) have recently been recognized as a
valuable and promising approach to discovering a policy with creditable
performance, particularly in the presence of a dynamic environment and
estimation errors in the transition matrix due to limited data. Despite
extensive exploration of dynamic programming algorithms for solving RMDPs,
there has been a notable upswing in interest in developing efficient algorithms
using the policy gradient method. In this paper, we propose the first
single-loop robust policy gradient (SRPG) method with the global optimality
guarantee for solving RMDPs through its minimax formulation. Moreover, we
complement the convergence analysis of the nonconvex-nonconcave min-max
optimization problem with the objective function's gradient dominance property,
which is not explored in the prior literature. Numerical experiments validate
the efficacy of SRPG, demonstrating its faster and more robust convergence
behavior compared to its nested-loop counterpart.
