  In this paper we present a framework for combining deep learning-based road
detection, particle filters, and Model Predictive Control (MPC) to drive
aggressively using only a monocular camera, IMU, and wheel speed sensors. This
framework uses deep convolutional neural networks combined with LSTMs to learn
a local cost map representation of the track in front of the vehicle. A
particle filter uses this dynamic observation model to localize in a schematic
map, and MPC is used to drive aggressively using this particle filter based
state estimate. We show extensive real world testing results, and demonstrate
reliable operation of the vehicle at the friction limits on a complex dirt
track. We reach speeds above 27 mph (12 m/s) on a dirt track with a 105 foot
(32m) long straight using our 1:5 scale test vehicle. A video of these results
can be found at https://www.youtube.com/watch?v=5ALIK-z-vUg
