  Compared to other language tasks, applying pre-trained language models (PLMs)
for search ranking often requires more nuances and training signals. In this
paper, we identify and study the two mismatches between pre-training and
ranking fine-tuning: the training schema gap regarding the differences in
training objectives and model architectures, and the task knowledge gap
considering the discrepancy between the knowledge needed in ranking and that
learned during pre-training. To mitigate these gaps, we propose Pre-trained,
Prompt-learned and Pre-finetuned Neural Ranker (P^3 Ranker). P^3 Ranker
leverages prompt-based learning to convert the ranking task into a pre-training
like schema and uses pre-finetuning to initialize the model on intermediate
supervised tasks. Experiments on MS MARCO and Robust04 show the superior
performances of P^3 Ranker in few-shot ranking. Analyses reveal that P^3 Ranker
is able to better accustom to the ranking task through prompt-based learning
and retrieve necessary ranking-oriented knowledge gleaned in pre-finetuning,
resulting in data-efficient PLM adaptation. Our code is available at
https://github.com/NEUIR/P3Ranker.
