  The progress in maritime obstacle detection is hindered by the lack of a
diverse dataset that adequately captures the complexity of general maritime
environments. We present the first maritime panoptic obstacle detection
benchmark LaRS, featuring scenes from Lakes, Rivers and Seas. Our major
contribution is the new dataset, which boasts the largest diversity in
recording locations, scene types, obstacle classes, and acquisition conditions
among the related datasets. LaRS is composed of over 4000 per-pixel labeled key
frames with nine preceding frames to allow utilization of the temporal texture,
amounting to over 40k frames. Each key frame is annotated with 8 thing, 3 stuff
classes and 19 global scene attributes. We report the results of 27 semantic
and panoptic segmentation methods, along with several performance insights and
future research directions. To enable objective evaluation, we have implemented
an online evaluation server. The LaRS dataset, evaluation toolkit and benchmark
are publicly available at: https://lojzezust.github.io/lars-dataset
