  In this paper, we present TWOLAR: a two-stage pipeline for passage reranking
based on the distillation of knowledge from Large Language Models (LLM). TWOLAR
introduces a new scoring strategy and a distillation process consisting in the
creation of a novel and diverse training dataset. The dataset consists of 20K
queries, each associated with a set of documents retrieved via four distinct
retrieval methods to ensure diversity, and then reranked by exploiting the
zero-shot reranking capabilities of an LLM. Our ablation studies demonstrate
the contribution of each new component we introduced. Our experimental results
show that TWOLAR significantly enhances the document reranking ability of the
underlying model, matching and in some cases even outperforming
state-of-the-art models with three orders of magnitude more parameters on the
TREC-DL test sets and the zero-shot evaluation benchmark BEIR. To facilitate
future work we release our data set, finetuned models, and code.
