  We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic
RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to
improve model hypothesis search, increasing the chance of finding outlier-free
minimal sets. Previous works use heuristic side-information like hand-crafted
descriptor distance to guide hypothesis search. In contrast, we learn
hypothesis search in a principled fashion that lets us optimize an arbitrary
task loss during training, leading to large improvements on classic computer
vision tasks. We present two further extensions to NG-RANSAC. Firstly, using
the inlier count itself as training signal allows us to train neural guidance
in a self-supervised fashion. Secondly, we combine neural guidance with
differentiable RANSAC to build neural networks which focus on certain parts of
the input data and make the output predictions as good as possible. We evaluate
NG-RANSAC on a wide array of computer vision tasks, namely estimation of
epipolar geometry, horizon line estimation and camera re-localization. We
achieve superior or competitive results compared to state-of-the-art robust
estimators, including very recent, learned ones.
