  This paper presents a new approach for integrating semantic information for
vision-based vehicle navigation. Although vision-based vehicle navigation
systems using pre-mapped visual landmarks are capable of achieving submeter
level accuracy in large-scale urban environment, a typical error source in this
type of systems comes from the presence of visual landmarks or features from
temporal objects in the environment, such as cars and pedestrians. We propose a
gated factor graph framework to use semantic information associated with visual
features to make decisions on outlier/ inlier computation from three
perspectives: the feature tracking process, the geo-referenced map building
process, and the navigation system using pre-mapped landmarks. The class
category that the visual feature belongs to is extracted from a pre-trained
deep learning network trained for semantic segmentation. The feasibility and
generality of our approach is demonstrated by our implementations on top of two
vision-based navigation systems. Experimental evaluations validate that the
injection of semantic information associated with visual landmarks using our
approach achieves substantial improvements in accuracy on GPS-denied navigation
solutions for large-scale urban scenarios
