  In this paper, we propose a method to partially mimic natural intelligence
for the problem of lifelong learning representations that are compatible. We
take the perspective of a learning agent that is interested in recognizing
object instances in an open dynamic universe in a way in which any update to
its internal feature representation does not render the features in the gallery
unusable for visual search. We refer to this learning problem as Compatible
Lifelong Learning Representations (CL2R) as it considers compatible
representation learning within the lifelong learning paradigm. We identify
stationarity as the property that the feature representation is required to
hold to achieve compatibility and propose a novel training procedure that
encourages local and global stationarity on the learned representation. Due to
stationarity, the statistical properties of the learned features do not change
over time, making them interoperable with previously learned features.
Extensive experiments on standard benchmark datasets show that our CL2R
training procedure outperforms alternative baselines and state-of-the-art
methods. We also provide novel metrics to specifically evaluate compatible
representation learning under catastrophic forgetting in various sequential
learning tasks. Code at
https://github.com/NiccoBiondi/CompatibleLifelongRepresentation.
