  Previous contrastive learning methods for sentence representations often
focus on insensitive transformations to produce positive pairs, but neglect the
role of sensitive transformations that are harmful to semantic representations.
Therefore, we propose an Equivariant Self-Contrastive Learning (ESCL) method to
make full use of sensitive transformations, which encourages the learned
representations to be sensitive to certain types of transformations with an
additional equivariant learning task. Meanwhile, in order to improve
practicability and generality, ESCL simplifies the implementations of
traditional equivariant contrastive methods to share model parameters from the
perspective of multi-task learning. We evaluate our ESCL on semantic textual
similarity tasks. The proposed method achieves better results while using fewer
learning parameters compared to previous methods.
