  Many applications, e.g., in shared mobility, require coordinating a large
number of agents. Mean-field reinforcement learning addresses the resulting
scalability challenge by optimizing the policy of a representative agent
interacting with the infinite population of identical agents instead of
considering individual pairwise interactions. In this paper, we address an
important generalization where there exist global constraints on the
distribution of agents (e.g., requiring capacity constraints or minimum
coverage requirements to be met). We propose Safe-M$^3$-UCRL, the first
model-based mean-field reinforcement learning algorithm that attains safe
policies even in the case of unknown transitions. As a key ingredient, it uses
epistemic uncertainty in the transition model within a log-barrier approach to
ensure pessimistic constraints satisfaction with high probability. Beyond the
synthetic swarm motion benchmark, we showcase Safe-M$^3$-UCRL on the vehicle
repositioning problem faced by many shared mobility operators and evaluate its
performance through simulations built on vehicle trajectory data from a service
provider in Shenzhen. Our algorithm effectively meets the demand in critical
areas while ensuring service accessibility in regions with low demand.
