  The primary goal of reinforcement learning is to develop decision-making
policies that prioritize optimal performance without considering risk or
safety. In contrast, safe reinforcement learning aims to mitigate or avoid
unsafe states. This paper presents a risk-sensitive Q-learning algorithm that
leverages optimal transport theory to enhance the agent safety. By integrating
optimal transport into the Q-learning framework, our approach seeks to optimize
the policy's expected return while minimizing the Wasserstein distance between
the policy's stationary distribution and a predefined risk distribution, which
encapsulates safety preferences from domain experts. We validate the proposed
algorithm in a Gridworld environment. The results indicate that our method
significantly reduces the frequency of visits to risky states and achieves
faster convergence to a stable policy compared to the traditional Q-learning
algorithm.
