  Current dense text retrieval models face two typical challenges. First, they
adopt a siamese dual-encoder architecture to encode queries and documents
independently for fast indexing and searching, while neglecting the
finer-grained term-wise interactions. This results in a sub-optimal recall
performance. Second, their model training highly relies on a negative sampling
technique to build up the negative documents in their contrastive losses. To
address these challenges, we present Adversarial Retriever-Ranker (AR2), which
consists of a dual-encoder retriever plus a cross-encoder ranker. The two
models are jointly optimized according to a minimax adversarial objective: the
retriever learns to retrieve negative documents to cheat the ranker, while the
ranker learns to rank a collection of candidates including both the
ground-truth and the retrieved ones, as well as providing progressive direct
feedback to the dual-encoder retriever. Through this adversarial game, the
retriever gradually produces harder negative documents to train a better
ranker, whereas the cross-encoder ranker provides progressive feedback to
improve retriever. We evaluate AR2 on three benchmarks. Experimental results
show that AR2 consistently and significantly outperforms existing dense
retriever methods and achieves new state-of-the-art results on all of them.
This includes the improvements on Natural Questions R@5 to 77.9%(+2.1%),
TriviaQA R@5 to 78.2%(+1.4), and MS-MARCO MRR@10 to 39.5%(+1.3%). Code and
models are available at https://github.com/microsoft/AR2.
