  Retrieval is a crucial stage in web search that identifies a small set of
query-relevant candidates from a billion-scale corpus. Discovering more
semantically-related candidates in the retrieval stage is very promising to
expose more high-quality results to the end users. However, it still remains
non-trivial challenges of building and deploying effective retrieval models for
semantic matching in real search engine. In this paper, we describe the
retrieval system that we developed and deployed in Baidu Search. The system
exploits the recent state-of-the-art Chinese pretrained language model, namely
Enhanced Representation through kNowledge IntEgration (ERNIE), which
facilitates the system with expressive semantic matching. In particular, we
developed an ERNIE-based retrieval model, which is equipped with 1) expressive
Transformer-based semantic encoders, and 2) a comprehensive multi-stage
training paradigm. More importantly, we present a practical system workflow for
deploying the model in web-scale retrieval. Eventually, the system is fully
deployed into production, where rigorous offline and online experiments were
conducted. The results show that the system can perform high-quality candidate
retrieval, especially for those tail queries with uncommon demands. Overall,
the new retrieval system facilitated by pretrained language model (i.e., ERNIE)
can largely improve the usability and applicability of our search engine.
