  Most existing robotic datasets capture static scene data and thus are limited
in evaluating robots' dynamic performance. To address this, we present a mobile
robot oriented large-scale indoor dataset, denoted as THUD (Tsinghua University
Dynamic) robotic dataset, for training and evaluating their dynamic scene
understanding algorithms. Specifically, the THUD dataset construction is first
detailed, including organization, acquisition, and annotation methods. It
comprises both real-world and synthetic data, collected with a real robot
platform and a physical simulation platform, respectively. Our current dataset
includes 13 larges-scale dynamic scenarios, 90K image frames, 20M 2D/3D
bounding boxes of static and dynamic objects, camera poses, and IMU. The
dataset is still continuously expanding. Then, the performance of mainstream
indoor scene understanding tasks, e.g. 3D object detection, semantic
segmentation, and robot relocalization, is evaluated on our THUD dataset. These
experiments reveal serious challenges for some robot scene understanding tasks
in dynamic scenes. By sharing this dataset, we aim to foster and iterate new
mobile robot algorithms quickly for robot actual working dynamic environment,
i.e. complex crowded dynamic scenes.
