  Central to the application of many multi-view geometry algorithms is the
extraction of matching points between multiple viewpoints, enabling classical
tasks such as camera pose estimation and 3D reconstruction. Many approaches
that characterize these points have been proposed based on hand-tuned
appearance models or data-driven learning methods. We propose Soft Expectation
and Deep Maximization (SEDM), an iterative unsupervised learning process that
directly optimizes the repeatability of the features by posing the problem in a
similar way to expectation maximization (EM). We found convergence to be
reliable and the new model to be more lighting invariant and better at localize
the underlying 3D points in a scene, improving SfM quality when compared to
other state of the art deep learning detectors.
