  Although neural information retrieval has witnessed great improvements,
recent works showed that the generalization ability of dense retrieval models
on target domains with different distributions is limited, which contrasts with
the results obtained with interaction-based models. To address this issue,
researchers have resorted to adversarial learning and query generation
approaches; both approaches nevertheless resulted in limited improvements. In
this paper, we propose to use a self-supervision approach in which
pseudo-relevance labels are automatically generated on the target domain. To do
so, we first use the standard BM25 model on the target domain to obtain a first
ranking of documents, and then use the interaction-based model T53B to re-rank
top documents. We further combine this approach with knowledge distillation
relying on an interaction-based teacher model trained on the source domain. Our
experiments reveal that pseudo-relevance labeling using T53B and the MiniLM
teacher performs on average better than other approaches and helps improve the
state-of-the-art query generation approach GPL when it is fine-tuned on the
pseudo-relevance labeled data.
