  Sample-and-rank is a key decoding strategy for modern generation-based
dialogue systems. It helps achieve diverse and high-quality responses by
selecting an answer from a small pool of generated candidates. The current
state-of-the-art ranking methods mainly use an encoding paradigm called
Cross-Encoder, which separately encodes each context-candidate pair and ranks
the candidates according to their fitness scores. However, Cross-Encoder
repeatedly encodes the same lengthy context for each candidate, resulting in
high computational costs. Poly-Encoder addresses the above problems by reducing
the interaction between context and candidates, but with a price of performance
drop. In this work, we develop a new paradigm called Uni-Encoder, that keeps
the full attention over each pair as in Cross-Encoder while only encoding the
context once, as in Poly-Encoder. Uni-Encoder encodes all the candidates with
the context in one forward pass. We use the same positional embedding for all
candidates to ensure they are treated equally and design a new attention
mechanism to avoid confusion. Our Uni-Encoder can simulate other ranking
paradigms using different attention and response concatenation methods.
Extensive experiments show that our proposed paradigm achieves new
state-of-the-art results on four benchmark datasets with high computational
efficiency. For instance, it improves R10@1 by 2.9% with an approximately 4X
faster inference speed on the Ubuntu V2 dataset.
