  The dual-encoder has become the de facto architecture for dense retrieval.
Typically, it computes the latent representations of the query and document
independently, thus failing to fully capture the interactions between the query
and document. To alleviate this, recent research has focused on obtaining
query-informed document representations. During training, it expands the
document with a real query, but during inference, it replaces the real query
with a generated one. This inconsistency between training and inference causes
the dense retrieval model to prioritize query information while disregarding
the document when computing the document representation. Consequently, it
performs even worse than the vanilla dense retrieval model because its
performance heavily relies on the relevance between the generated queries and
the real query.In this paper, we propose a curriculum sampling strategy that
utilizes pseudo queries during training and progressively enhances the
relevance between the generated query and the real query. By doing so, the
retrieval model learns to extend its attention from the document alone to both
the document and query, resulting in high-quality query-informed document
representations. Experimental results on both in-domain and out-of-domain
datasets demonstrate that our approach outperforms previous dense retrieval
models.
