  To achieve a driverless train operation on mainline railways, actual and
potential obstacles for the train's driveway must be detected automatically by
appropriate sensor systems. Machine learning algorithms have proven to be
powerful tools for this task during the last years. However, these algorithms
require large amounts of high-quality annotated data containing
railway-specific objects as training data. Unfortunately, all of the publicly
available datasets that tackle this requirement are restricted in some way.
Therefore, this paper presents OSDaR23, a multi-sensor dataset of 45
subsequences acquired in Hamburg, Germany, in September 2021, that was created
to foster driverless train operation on mainline railways. The sensor setup
consists of multiple calibrated and synchronized infrared (IR) and visual (RGB)
cameras, lidars, a radar, and position and acceleration sensors mounted on the
front of a rail vehicle. In addition to the raw data, the dataset contains
204091 polyline, polygonal, rectangle, and cuboid annotations in total for 20
different object classes. It is the first publicly available multi-sensor
dataset annotated with a variety of object classes that are relevant for the
railway context. OSDaR23, available at data.fid-move.de/dataset/osdar23, can
also be used for tasks beyond collision prediction, which are listed in this
paper.
