  We introduce the UT Campus Object Dataset (CODa), a mobile robot egocentric
perception dataset collected on the University of Texas Austin Campus. Our
dataset contains 8.5 hours of multimodal sensor data: synchronized 3D point
clouds and stereo RGB video from a 128-channel 3D LiDAR and two 1.25MP RGB
cameras at 10 fps; RGB-D videos from an additional 0.5MP sensor at 7 fps, and a
9-DOF IMU sensor at 40 Hz. We provide 58 minutes of ground-truth annotations
containing 1.3 million 3D bounding boxes with instance IDs for 53 semantic
classes, 5000 frames of 3D semantic annotations for urban terrain, and
pseudo-ground truth localization. We repeatedly traverse identical geographic
locations for a wide range of indoor and outdoor areas, weather conditions, and
times of the day. Using CODa, we empirically demonstrate that: 1) 3D object
detection performance in urban settings is significantly higher when trained
using CODa compared to existing datasets even when employing state-of-the-art
domain adaptation approaches, 2) sensor-specific fine-tuning improves 3D object
detection accuracy and 3) pretraining on CODa improves cross-dataset 3D object
detection performance in urban settings compared to pretraining on AV datasets.
Using our dataset and annotations, we release benchmarks for 3D object
detection and 3D semantic segmentation using established metrics. In the
future, the CODa benchmark will include additional tasks like unsupervised
object discovery and re-identification. We publicly release CODa on the Texas
Data Repository, pre-trained models, dataset development package, and
interactive dataset viewer on our website at https://amrl.cs.utexas.edu/coda.
We expect CODa to be a valuable dataset for research in egocentric 3D
perception and planning for autonomous navigation in urban environments.
