  State-of-the-art autonomous driving systems rely on high definition (HD) maps
for localization and navigation. However, building and maintaining HD maps is
time-consuming and expensive. Furthermore, the HD maps assume structured
environment such as the existence of major road and lanes, which are not
present in rural areas. In this work, we propose an end-to-end transformer
networks based approach for map-less autonomous driving. The proposed model
takes raw LiDAR data and noisy topometric map as input and produces precise
local trajectory for navigation. We demonstrate the effectiveness of our method
in real-world driving data, including both urban and rural areas. The
experimental results show that the proposed method outperforms state-of-the-art
multimodal methods and is robust to the perturbations of the topometric map.
The code of the proposed method is publicly available at
\url{https://github.com/Jiaolong/trajectory-prediction}.
