  Most sentence embedding techniques heavily rely on expensive human-annotated
sentence pairs as the supervised signals. Despite the use of large-scale
unlabeled data, the performance of unsupervised methods typically lags far
behind that of the supervised counterparts in most downstream tasks. In this
work, we propose a semi-supervised sentence embedding framework, GenSE, that
effectively leverages large-scale unlabeled data. Our method include three
parts: 1) Generate: A generator/discriminator model is jointly trained to
synthesize sentence pairs from open-domain unlabeled corpus; 2) Discriminate:
Noisy sentence pairs are filtered out by the discriminator to acquire
high-quality positive and negative sentence pairs; 3) Contrast: A prompt-based
contrastive approach is presented for sentence representation learning with
both annotated and synthesized data. Comprehensive experiments show that GenSE
achieves an average correlation score of 85.19 on the STS datasets and
consistent performance improvement on four domain adaptation tasks,
significantly surpassing the state-of-the-art methods and convincingly
corroborating its effectiveness and generalization ability.Code, Synthetic data
and Models available at https://github.com/MatthewCYM/GenSE.
