  We present TartanDrive, a large scale dataset for learning dynamics models
for off-road driving. We collected a dataset of roughly 200,000 off-road
driving interactions on a modified Yamaha Viking ATV with seven unique sensing
modalities in diverse terrains. To the authors' knowledge, this is the largest
real-world multi-modal off-road driving dataset, both in terms of number of
interactions and sensing modalities. We also benchmark several state-of-the-art
methods for model-based reinforcement learning from high-dimensional
observations on this dataset. We find that extending these models to
multi-modality leads to significant performance on off-road dynamics
prediction, especially in more challenging terrains. We also identify some
shortcomings with current neural network architectures for the off-road driving
task. Our dataset is available at https://github.com/castacks/tartan_drive.
