  Autonomous driving involves complex decision-making in highly interactive
environments, requiring thoughtful negotiation with other traffic participants.
While reinforcement learning provides a way to learn such interaction behavior,
efficient learning critically depends on scalable state representations.
Contrary to imitation learning methods, high-dimensional state representations
still constitute a major bottleneck for deep reinforcement learning methods in
autonomous driving. In this paper, we study the challenges of constructing
bird's-eye-view representations for autonomous driving and propose a recurrent
learning architecture for long-horizon driving. Our PPO-based approach, called
RecurrDriveNet, is demonstrated on a simulated autonomous driving task in
CARLA, where it outperforms traditional frame-stacking methods while only
requiring one million experiences for efficient training. RecurrDriveNet causes
less than one infraction per driven kilometer by interacting safely with other
road users.
