  Unsupervised optical flow estimation is especially hard near occlusions and
motion boundaries and in low-texture regions. We show that additional
information such as semantics and domain knowledge can help better constrain
this problem. We introduce SemARFlow, an unsupervised optical flow network
designed for autonomous driving data that takes estimated semantic segmentation
masks as additional inputs. This additional information is injected into the
encoder and into a learned upsampler that refines the flow output. In addition,
a simple yet effective semantic augmentation module provides self-supervision
when learning flow and its boundaries for vehicles, poles, and sky. Together,
these injections of semantic information improve the KITTI-2015 optical flow
test error rate from 11.80% to 8.38%. We also show visible improvements around
object boundaries as well as a greater ability to generalize across datasets.
Code is available at
https://github.com/duke-vision/semantic-unsup-flow-release.
