  Recent works have shown that optical flow can be learned by deep networks
from unlabelled image pairs based on brightness constancy assumption and
smoothness prior. Current approaches additionally impose an augmentation
regularization term for continual self-supervision, which has been proved to be
effective on difficult matching regions. However, this method also amplify the
inevitable mismatch in unsupervised setting, blocking the learning process
towards optimal solution. To break the dilemma, we propose a novel mutual
distillation framework to transfer reliable knowledge back and forth between
the teacher and student networks for alternate improvement. Concretely, taking
estimation of off-the-shelf unsupervised approach as pseudo labels, our insight
locates at defining a confidence selection mechanism to extract relative good
matches, and then add diverse data augmentation for distilling adequate and
reliable knowledge from teacher to student. Thanks to the decouple nature of
our method, we can choose a stronger student architecture for sufficient
learning. Finally, better student prediction is adopted to transfer knowledge
back to the efficient teacher without additional costs in real deployment.
Rather than formulating it as a supervised task, we find that introducing an
extra unsupervised term for multi-target learning achieves best final results.
Extensive experiments show that our approach, termed MDFlow, achieves
state-of-the-art real-time accuracy and generalization ability on challenging
benchmarks. Code is available at https://github.com/ltkong218/MDFlow.
