  Time-series representation learning can extract representations from data
with temporal dynamics and sparse labels. When labeled data are sparse but
unlabeled data are abundant, contrastive learning, i.e., a framework to learn a
latent space where similar samples are close to each other while dissimilar
ones are far from each other, has shown outstanding performance. This strategy
can encourage varied consistency of time-series representations depending on
the positive pair selection and contrastive loss. We propose a new time-series
representation learning method by combining the advantages of self-supervised
tasks related to contextual, temporal, and transformation consistency. It
allows the network to learn general representations for various downstream
tasks and domains. Specifically, we first adopt data preprocessing to generate
positive and negative pairs for each self-supervised task. The model then
performs contextual, temporal, and transformation contrastive learning and is
optimized jointly using their contrastive losses. We further investigate an
uncertainty weighting approach to enable effective multi-task learning by
considering the contribution of each consistency. We evaluate the proposed
framework on three downstream tasks: time-series classification, forecasting,
and anomaly detection. Experimental results show that our method not only
outperforms the benchmark models on these downstream tasks, but also shows
efficiency in cross-domain transfer learning.
