  We propose a neuralized undirected graphical model called Neural-Hidden-CRF
to solve the weakly-supervised sequence labeling problem. Under the umbrella of
probabilistic undirected graph theory, the proposed Neural-Hidden-CRF embedded
with a hidden CRF layer models the variables of word sequence, latent ground
truth sequence, and weak label sequence with the global perspective that
undirected graphical models particularly enjoy. In Neural-Hidden-CRF, we can
capitalize on the powerful language model BERT or other deep models to provide
rich contextual semantic knowledge to the latent ground truth sequence, and use
the hidden CRF layer to capture the internal label dependencies.
Neural-Hidden-CRF is conceptually simple and empirically powerful. It obtains
new state-of-the-art results on one crowdsourcing benchmark and three
weak-supervision benchmarks, including outperforming the recent advanced model
CHMM by 2.80 F1 points and 2.23 F1 points in average generalization and
inference performance, respectively.
