  Simultaneous Localization and Mapping (SLAM) is one of the most essential
techniques in many real-world robotic applications. The assumption of static
environments is common in most SLAM algorithms, which however, is not the case
for most applications. Recent work on semantic SLAM aims to understand the
objects in an environment and distinguish dynamic information from a scene
context by performing image-based segmentation. However, the segmentation
results are often imperfect or incomplete, which can subsequently reduce the
quality of mapping and the accuracy of localization. In this paper, we present
a robust multi-modal semantic framework to solve the SLAM problem in complex
and highly dynamic environments. We propose to learn a more powerful object
feature representation and deploy the mechanism of looking and thinking twice
to the backbone network, which leads to a better recognition result to our
baseline instance segmentation model. Moreover, both geometric-only clustering
and visual semantic information are combined to reduce the effect of
segmentation error due to small-scale objects, occlusion and motion blur.
Thorough experiments have been conducted to evaluate the performance of the
proposed method. The results show that our method can precisely identify
dynamic objects under recognition imperfection and motion blur. Moreover, the
proposed SLAM framework is able to efficiently build a static dense map at a
processing rate of more than 10 Hz, which can be implemented in many practical
applications. Both training data and the proposed method is open sourced at
https://github.com/wh200720041/MMS_SLAM.
