  We present a system that enables an autonomous small-scale RC car to drive
aggressively from visual observations using reinforcement learning (RL). Our
system, FastRLAP (faster lap), trains autonomously in the real world, without
human interventions, and without requiring any simulation or expert
demonstrations. Our system integrates a number of important components to make
this possible: we initialize the representations for the RL policy and value
function from a large prior dataset of other robots navigating in other
environments (at low speed), which provides a navigation-relevant
representation. From here, a sample-efficient online RL method uses a single
low-speed user-provided demonstration to determine the desired driving course,
extracts a set of navigational checkpoints, and autonomously practices driving
through these checkpoints, resetting automatically on collision or failure.
Perhaps surprisingly, we find that with appropriate initialization and choice
of algorithm, our system can learn to drive over a variety of racing courses
with less than 20 minutes of online training. The resulting policies exhibit
emergent aggressive driving skills, such as timing braking and acceleration
around turns and avoiding areas which impede the robot's motion, approaching
the performance of a human driver using a similar first-person interface over
the course of training.
