  While learning visuomotor skills in an end-to-end manner is appealing, deep
neural networks are often uninterpretable and fail in surprising ways. For
robotics tasks, such as autonomous driving, models that explicitly represent
objects may be more robust to new scenes and provide intuitive visualizations.
We describe a taxonomy of "object-centric" models which leverage both object
instances and end-to-end learning. In the Grand Theft Auto V simulator, we show
that object-centric models outperform object-agnostic methods in scenes with
other vehicles and pedestrians, even with an imperfect detector. We also
demonstrate that our architectures perform well on real-world environments by
evaluating on the Berkeley DeepDrive Video dataset, where an object-centric
model outperforms object-agnostic models in the low-data regimes.
