  Robust and accurate localization is an essential component for robotic
navigation and autonomous driving. The use of cameras for localization with
high definition map (HD Map) provides an affordable localization sensor set.
Existing methods suffer from pose estimation failure due to error prone data
association or initialization with accurate initial pose requirement. In this
paper, we propose a cost-effective vehicle localization system with HD map for
autonomous driving that uses cameras as primary sensors. To this end, we
formulate vision-based localization as a data association problem that maps
visual semantics to landmarks in HD map. Specifically, system initialization is
finished in a coarse to fine manner by combining coarse GPS (Global Positioning
System) measurement and fine pose searching. In tracking stage, vehicle pose is
refined by implicitly aligning the semantic segmentation result between image
and landmarks in HD maps with photometric consistency. Finally, vehicle pose is
computed by pose graph optimization in a sliding window fashion. We evaluate
our method on two datasets and demonstrate that the proposed approach yields
promising localization results in different driving scenarios. Additionally,
our approach is suitable for both monocular camera and multi-cameras that
provides flexibility and improves robustness for the localization system.
