  Operating a robot in the open world requires a high level of robustness with
respect to previously unseen environments. Optimally, the robot is able to
adapt by itself to new conditions without human supervision, e.g.,
automatically adjusting its perception system to changing lighting conditions.
In this work, we address the task of continual learning for deep learning-based
monocular depth estimation and panoptic segmentation in new environments in an
online manner. We introduce CoDEPS to perform continual learning involving
multiple real-world domains while mitigating catastrophic forgetting by
leveraging experience replay. In particular, we propose a novel domain-mixing
strategy to generate pseudo-labels to adapt panoptic segmentation. Furthermore,
we explicitly address the limited storage capacity of robotic systems by
leveraging sampling strategies for constructing a fixed-size replay buffer
based on rare semantic class sampling and image diversity. We perform extensive
evaluations of CoDEPS on various real-world datasets demonstrating that it
successfully adapts to unseen environments without sacrificing performance on
previous domains while achieving state-of-the-art results. The code of our work
is publicly available at http://codeps.cs.uni-freiburg.de.
