  Perception-driven approach and end-to-end system are two major vision-based
frameworks for self-driving cars. However, it is difficult to introduce
attention and historical information of autonomous driving process, which are
the essential factors for achieving human-like driving into these two methods.
In this paper, we propose a novel model for self-driving cars named
brain-inspired cognitive model with attention (CMA). This model consists of
three parts: a convolutional neural network for simulating human visual cortex,
a cognitive map built to describe relationships between objects in complex
traffic scene and a recurrent neural network that combines with the real-time
updated cognitive map to implement attention mechanism and long-short term
memory. The benefit of our model is that can accurately solve three tasks
simultaneously:1) detection of the free space and boundaries of the current and
adjacent lanes. 2)estimation of obstacle distance and vehicle attitude, and 3)
learning of driving behavior and decision making from human driver. More
significantly, the proposed model could accept external navigating instructions
during an end-to-end driving process. For evaluation, we build a large-scale
road-vehicle dataset which contains more than forty thousand labeled road
images captured by three cameras on our self-driving car. Moreover, human
driving activities and vehicle states are recorded in the meanwhile.
