  Sequence labeling systems should perform reliably not only under ideal
conditions but also with corrupted inputs - as these systems often process
user-generated text or follow an error-prone upstream component. To this end,
we formulate the noisy sequence labeling problem, where the input may undergo
an unknown noising process and propose two Noise-Aware Training (NAT)
objectives that improve robustness of sequence labeling performed on perturbed
input: Our data augmentation method trains a neural model using a mixture of
clean and noisy samples, whereas our stability training algorithm encourages
the model to create a noise-invariant latent representation. We employ a
vanilla noise model at training time. For evaluation, we use both the original
data and its variants perturbed with real OCR errors and misspellings.
Extensive experiments on English and German named entity recognition benchmarks
confirmed that NAT consistently improved robustness of popular sequence
labeling models, preserving accuracy on the original input. We make our code
and data publicly available for the research community.
