  Not all positive pairs are beneficial to time series contrastive learning. In
this paper, we study two types of bad positive pairs that can impair the
quality of time series representation learned through contrastive learning: the
noisy positive pair and the faulty positive pair. We observe that, with the
presence of noisy positive pairs, the model tends to simply learn the pattern
of noise (Noisy Alignment). Meanwhile, when faulty positive pairs arise, the
model wastes considerable amount of effort aligning non-representative patterns
(Faulty Alignment). To address this problem, we propose a Dynamic Bad Pair
Mining (DBPM) algorithm, which reliably identifies and suppresses bad positive
pairs in time series contrastive learning. Specifically, DBPM utilizes a memory
module to dynamically track the training behavior of each positive pair along
training process. This allows us to identify potential bad positive pairs at
each epoch based on their historical training behaviors. The identified bad
pairs are subsequently down-weighted through a transformation module, thereby
mitigating their negative impact on the representation learning process. DBPM
is a simple algorithm designed as a lightweight plug-in without learnable
parameters to enhance the performance of existing state-of-the-art methods.
Through extensive experiments conducted on four large-scale, real-world time
series datasets, we demonstrate DBPM's efficacy in mitigating the adverse
effects of bad positive pairs.
