  Boundary information is critical for various Chinese language processing
tasks, such as word segmentation, part-of-speech tagging, and named entity
recognition. Previous studies usually resorted to the use of a high-quality
external lexicon, where lexicon items can offer explicit boundary information.
However, to ensure the quality of the lexicon, great human effort is always
necessary, which has been generally ignored. In this work, we suggest
unsupervised statistical boundary information instead, and propose an
architecture to encode the information directly into pre-trained language
models, resulting in Boundary-Aware BERT (BABERT). We apply BABERT for feature
induction of Chinese sequence labeling tasks. Experimental results on ten
benchmarks of Chinese sequence labeling demonstrate that BABERT can provide
consistent improvements on all datasets. In addition, our method can complement
previous supervised lexicon exploration, where further improvements can be
achieved when integrated with external lexicon information.
