  Recent studies have demonstrated the effectiveness of using large language
language models (LLMs) in passage ranking. The listwise approaches, such as
RankGPT, have become new state-of-the-art in this task. However, the efficiency
of RankGPT models is limited by the maximum context length and relatively high
latency of LLM inference. To address these issues, in this paper, we propose
PE-Rank, leveraging the single passage embedding as a good context compression
for efficient listwise passage reranking. By treating each passage as a special
token, we can directly input passage embeddings into LLMs, thereby reducing
input length. Additionally, we introduce an inference method that dynamically
constrains the decoding space to these special tokens, accelerating the
decoding process. For adapting the model to reranking, we employ listwise
learning to rank loss for training. Evaluation results on multiple benchmarks
demonstrate that PE-Rank significantly improves efficiency in both prefilling
and decoding, while maintaining competitive ranking effectiveness. {The Code is
available at \url{https://github.com/liuqi6777/pe_rank}.}
