  Ranker and retriever are two important components in dense passage retrieval.
The retriever typically adopts a dual-encoder model, where queries and
documents are separately input into two pre-trained models, and the vectors
generated by the models are used for similarity calculation. The ranker often
uses a cross-encoder model, where the concatenated query-document pairs are
input into a pre-trained model to obtain word similarities. However, the
dual-encoder model lacks interaction between queries and documents due to its
independent encoding, while the cross-encoder model requires substantial
computational cost for attention calculation, making it difficult to obtain
real-time retrieval results. In this paper, we propose a dense retrieval model
called MD2PR based on multi-level distillation. In this model, we distill the
knowledge learned from the cross-encoder to the dual-encoder at both the
sentence level and word level. Sentence-level distillation enhances the
dual-encoder on capturing the themes and emotions of sentences. Word-level
distillation improves the dual-encoder in analysis of word semantics and
relationships. As a result, the dual-encoder can be used independently for
subsequent encoding and retrieval, avoiding the significant computational cost
associated with the participation of the cross-encoder. Furthermore, we propose
a simple dynamic filtering method, which updates the threshold during multiple
training iterations to ensure the effective identification of false negatives
and thus obtains a more comprehensive semantic representation space. The
experimental results over two standard datasets show our MD2PR outperforms 11
baseline models in terms of MRR and Recall metrics.
