  First-stage retrieval is a critical task that aims to retrieve relevant
document candidates from a large-scale collection. While existing retrieval
models have achieved impressive performance, they are mostly studied on static
data sets, ignoring that in the real-world, the data on the Web is continuously
growing with potential distribution drift. Consequently, retrievers trained on
static old data may not suit new-coming data well and inevitably produce
sub-optimal results. In this work, we study lifelong learning for first-stage
retrieval, especially focusing on the setting where the emerging documents are
unlabeled since relevance annotation is expensive and may not keep up with data
emergence. Under this setting, we aim to develop model updating with two goals:
(1) to effectively adapt to the evolving distribution with the unlabeled
new-coming data, and (2) to avoid re-inferring all embeddings of old documents
to efficiently update the index each time the model is updated.
  We first formalize the task and then propose a novel Lifelong Learning method
for the first-stage Retrieval, namely L^2R. L^2R adopts the typical memory
mechanism for lifelong learning, and incorporates two crucial components: (1)
selecting diverse support negatives for model training and memory updating for
effective model adaptation, and (2) a ranking alignment objective to ensure the
backward-compatibility of representations to save the cost of index rebuilding
without hurting the model performance. For evaluation, we construct two new
benchmarks from LoTTE and Multi-CPR datasets to simulate the document
distribution drift in realistic retrieval scenarios. Extensive experiments show
that L^2R significantly outperforms competitive lifelong learning baselines.
