  Dashboard cameras capture a tremendous amount of driving scene video each
day. These videos are purposefully coupled with vehicle sensing data, such as
from the speedometer and inertial sensors, providing an additional sensing
modality for free. In this work, we leverage the large-scale unlabeled yet
naturally paired data for visual representation learning in the driving
scenario. A representation is learned in an end-to-end self-supervised
framework for predicting dense optical flow from a single frame with paired
sensing data. We postulate that success on this task requires the network to
learn semantic and geometric knowledge in the ego-centric view. For example,
forecasting a future view to be seen from a moving vehicle requires an
understanding of scene depth, scale, and movement of objects. We demonstrate
that our learned representation can benefit other tasks that require detailed
scene understanding and outperforms competing unsupervised representations on
semantic segmentation.
