  Vision-based autonomous urban driving in dense traffic is quite challenging
due to the complicated urban environment and the dynamics of the driving
behaviors. Widely-applied methods either heavily rely on hand-crafted rules or
learn from limited human experience, which makes them hard to generalize to
rare but critical scenarios. In this paper, we present a novel CAscade Deep
REinforcement learning framework, CADRE, to achieve model-free vision-based
autonomous urban driving. In CADRE, to derive representative latent features
from raw observations, we first offline train a Co-attention Perception Module
(CoPM) that leverages the co-attention mechanism to learn the
inter-relationships between the visual and control information from a
pre-collected driving dataset. Cascaded by the frozen CoPM, we then present an
efficient distributed proximal policy optimization framework to online learn
the driving policy under the guidance of particularly designed reward
functions. We perform a comprehensive empirical study with the CARLA NoCrash
benchmark as well as specific obstacle avoidance scenarios in autonomous urban
driving tasks. The experimental results well justify the effectiveness of CADRE
and its superiority over the state-of-the-art by a wide margin.
