  Enhancing diverse human decision-making processes in an urban environment is
a critical issue across various applications, including ride-sharing vehicle
dispatching, public transportation management, and autonomous driving. Offline
reinforcement learning (RL) is a promising approach to learn and optimize human
urban strategies (or policies) from pre-collected human-generated
spatial-temporal urban data. However, standard offline RL faces two significant
challenges: (1) data scarcity and data heterogeneity, and (2) distributional
shift. In this paper, we introduce MODA -- a Multi-Task Offline Reinforcement
Learning with Contrastive Data Sharing approach. MODA addresses the challenges
of data scarcity and heterogeneity in a multi-task urban setting through
Contrastive Data Sharing among tasks. This technique involves extracting latent
representations of human behaviors by contrasting positive and negative data
pairs. It then shares data presenting similar representations with the target
task, facilitating data augmentation for each task. Moreover, MODA develops a
novel model-based multi-task offline RL algorithm. This algorithm constructs a
robust Markov Decision Process (MDP) by integrating a dynamics model with a
Generative Adversarial Network (GAN). Once the robust MDP is established, any
online RL or planning algorithm can be applied. Extensive experiments conducted
in a real-world multi-task urban setting validate the effectiveness of MODA.
The results demonstrate that MODA exhibits significant improvements compared to
state-of-the-art baselines, showcasing its capability in advancing urban
decision-making processes. We also made our code available to the research
community.
