  The performance of state-of-the-art neural rankers can deteriorate
substantially when exposed to noisy inputs or applied to a new domain. In this
paper, we present a novel method for fine-tuning neural rankers that can
significantly improve their robustness to out-of-domain data and query
perturbations. Specifically, a contrastive loss that compares data points in
the representation space is combined with the standard ranking loss during
fine-tuning. We use relevance labels to denote similar/dissimilar pairs, which
allows the model to learn the underlying matching semantics across different
query-document pairs and leads to improved robustness. In experiments with four
passage ranking datasets, the proposed contrastive fine-tuning method obtains
improvements on robustness to query reformulations, noise perturbations, and
zero-shot transfer for both BERT and BART based rankers. Additionally, our
experiments show that contrastive fine-tuning outperforms data augmentation for
robustifying neural rankers.
