  Learning universal time series representations applicable to various types of
downstream tasks is challenging but valuable in real applications. Recently,
researchers have attempted to leverage the success of self-supervised
contrastive learning (SSCL) in Computer Vision(CV) and Natural Language
Processing(NLP) to tackle time series representation. Nevertheless, due to the
special temporal characteristics, relying solely on empirical guidance from
other domains may be ineffective for time series and difficult to adapt to
multiple downstream tasks. To this end, we review three parts involved in SSCL
including 1) designing augmentation methods for positive pairs, 2) constructing
(hard) negative pairs, and 3) designing SSCL loss. For 1) and 2), we find that
unsuitable positive and negative pair construction may introduce inappropriate
inductive biases, which neither preserve temporal properties nor provide
sufficient discriminative features. For 3), just exploring segment- or
instance-level semantics information is not enough for learning universal
representation. To remedy the above issues, we propose a novel self-supervised
framework named TimesURL. Specifically, we first introduce a
frequency-temporal-based augmentation to keep the temporal property unchanged.
And then, we construct double Universums as a special kind of hard negative to
guide better contrastive learning. Additionally, we introduce time
reconstruction as a joint optimization objective with contrastive learning to
capture both segment-level and instance-level information. As a result,
TimesURL can learn high-quality universal representations and achieve
state-of-the-art performance in 6 different downstream tasks, including short-
and long-term forecasting, imputation, classification, anomaly detection and
transfer learning.
