  Recently, contrastive learning has been shown to be effective in improving
pre-trained language models (PLM) to derive high-quality sentence
representations. It aims to pull close positive examples to enhance the
alignment while push apart irrelevant negatives for the uniformity of the whole
representation space. However, previous works mostly adopt in-batch negatives
or sample from training data at random. Such a way may cause the sampling bias
that improper negatives (e.g. false negatives and anisotropy representations)
are used to learn sentence representations, which will hurt the uniformity of
the representation space. To address it, we present a new framework
\textbf{DCLR} (\underline{D}ebiased \underline{C}ontrastive
\underline{L}earning of unsupervised sentence \underline{R}epresentations) to
alleviate the influence of these improper negatives. In DCLR, we design an
instance weighting method to punish false negatives and generate noise-based
negatives to guarantee the uniformity of the representation space. Experiments
on seven semantic textual similarity tasks show that our approach is more
effective than competitive baselines. Our code and data are publicly available
at the link: \textcolor{blue}{\url{https://github.com/RUCAIBox/DCLR}}.
