  Pair-based metric learning has been widely adopted to learn sentence
embedding in many NLP tasks such as semantic text similarity due to its
efficiency in computation. Most existing works employed a sequence encoder
model and utilized limited sentence pairs with a pair-based loss to learn
discriminating sentence representation. However, it is known that the sentence
representation can be biased when the sampled sentence pairs deviate from the
true distribution of all sentence pairs. In this paper, our theoretical
analysis shows that existing works severely suffered from a good pair sampling
and instance weighting strategy. Instead of one time pair selection and
learning on equal weighted pairs, we propose a unified locality weighting and
learning framework to learn task-specific sentence embedding. Our model,
SentPWNet, exploits the neighboring spatial distribution of each sentence as
locality weight to indicate the informative level of sentence pair. Such weight
is updated along with pair-loss optimization in each round, ensuring the model
keep learning the most informative sentence pairs. Extensive experiments on
four public available datasets and a self-collected place search benchmark with
1.4 million places clearly demonstrate that our model consistently outperforms
existing sentence embedding methods with comparable efficiency.
