  A safe and robust on-road navigation system is a crucial component of
achieving fully automated vehicles. NVIDIA recently proposed an End-to-End
algorithm that can directly learn steering commands from raw pixels of a front
camera by using one convolutional neural network. In this paper, we leverage
auxiliary information aside from raw images and design a novel network
structure, called Auxiliary Task Network (ATN), to help boost the driving
performance while maintaining the advantage of minimal training data and an
End-to-End training method. In this network, we introduce human prior knowledge
into vehicle navigation by transferring features from image recognition tasks.
Image semantic segmentation is applied as an auxiliary task for navigation. We
consider temporal information by introducing an LSTM module and optical flow to
the network. Finally, we combine vehicle kinematics with a sensor fusion step.
We discuss the benefits of our method over state-of-the-art visual navigation
methods both in the Udacity simulation environment and on the real-world
Comma.ai dataset.
