  Document retrieval has been extensively studied within the index-retrieve
framework for decades, which has withstood the test of time. Unfortunately,
such a pipelined framework limits the optimization of the final retrieval
quality, because indexing and retrieving are separated stages that can not be
jointly optimized in an end-to-end manner. In order to unify these two stages,
we explore a model-based indexer for document retrieval. Concretely, we propose
Ultron, which encodes the knowledge of all documents into the model and aims to
directly retrieve relevant documents end-to-end. For the model-based indexer,
how to represent docids and how to train the model are two main issues to be
explored. Existing solutions suffer from semantically deficient docids and
limited supervised data. To tackle these two problems, first, we devise two
types of docids that are richer in semantics and easier for model inference. In
addition, we propose a three-stage training workflow to capture more knowledge
contained in the corpus and associations between queries and docids.
Experiments on two public datasets demonstrate the superiority of Ultron over
advanced baselines for document retrieval.
