  Recent advances in deep learning for edge detection and segmentation opens up
a new path for semantic-edge-based ego-motion estimation. In this work, we
propose a robust monocular visual odometry (VO) framework using category-aware
semantic edges. It can reconstruct large-scale semantic maps in challenging
outdoor environments. The core of our approach is a semantic nearest neighbor
field that facilitates a robust data association of edges across frames using
semantics. This significantly enlarges the convergence radius during tracking
phases. The proposed edge registration method can be easily integrated into
direct VO frameworks to estimate photometrically, geometrically, and
semantically consistent camera motions. Different types of edges are evaluated
and extensive experiments demonstrate that our proposed system outperforms
state-of-art indirect, direct, and semantic monocular VO systems.
