  To safely navigate intricate real-world scenarios, autonomous vehicles must
be able to adapt to diverse road conditions and anticipate future events. World
model (WM) based reinforcement learning (RL) has emerged as a promising
approach by learning and predicting the complex dynamics of various
environments. Nevertheless, to the best of our knowledge, there does not exist
an accessible platform for training and testing such algorithms in
sophisticated driving environments. To fill this void, we introduce CarDreamer,
the first open-source learning platform designed specifically for developing WM
based autonomous driving algorithms. It comprises three key components: 1)
World model backbone: CarDreamer has integrated some state-of-the-art WMs,
which simplifies the reproduction of RL algorithms. The backbone is decoupled
from the rest and communicates using the standard Gym interface, so that users
can easily integrate and test their own algorithms. 2) Built-in tasks:
CarDreamer offers a comprehensive set of highly configurable driving tasks
which are compatible with Gym interfaces and are equipped with empirically
optimized reward functions. 3) Task development suite: This suite streamlines
the creation of driving tasks, enabling easy definition of traffic flows and
vehicle routes, along with automatic collection of multi-modal observation
data. A visualization server allows users to trace real-time agent driving
videos and performance metrics through a browser. Furthermore, we conduct
extensive experiments using built-in tasks to evaluate the performance and
potential of WMs in autonomous driving. Thanks to the richness and flexibility
of CarDreamer, we also systematically study the impact of observation modality,
observability, and sharing of vehicle intentions on AV safety and efficiency.
All code and documents are accessible on
https://github.com/ucd-dare/CarDreamer.
