  We introduce a compact network for holistic scene flow estimation, called
SENSE, which shares common encoder features among four closely-related tasks:
optical flow estimation, disparity estimation from stereo, occlusion
estimation, and semantic segmentation. Our key insight is that sharing features
makes the network more compact, induces better feature representations, and can
better exploit interactions among these tasks to handle partially labeled data.
With a shared encoder, we can flexibly add decoders for different tasks during
training. This modular design leads to a compact and efficient model at
inference time. Exploiting the interactions among these tasks allows us to
introduce distillation and self-supervised losses in addition to supervised
losses, which can better handle partially labeled real-world data. SENSE
achieves state-of-the-art results on several optical flow benchmarks and runs
as fast as networks specifically designed for optical flow. It also compares
favorably against the state of the art on stereo and scene flow, while
consuming much less memory.
