  Part of speech tagging is a fundamental NLP task often regarded as solved for
high-resource languages such as English. Current state-of-the-art models have
achieved high accuracy, especially on the news domain. However, when these
models are applied to other corpora with different genres, and especially
user-generated data from the Web, we see substantial drops in performance. In
this work, we study how a state-of-the-art tagging model trained on different
genres performs on Web content from unfiltered Reddit forum discussions. More
specifically, we use data from multiple sources: OntoNotes, a large benchmark
corpus with 'well-edited' text, the English Web Treebank with 5 Web genres, and
GUM, with 7 further genres other than Reddit. We report the results when
training on different splits of the data, tested on Reddit. Our results show
that even small amounts of in-domain data can outperform the contribution of
data an order of magnitude larger coming from other Web domains. To make
progress on out-of-domain tagging, we also evaluate an ensemble approach using
multiple single-genre taggers as input features to a meta-classifier. We
present state of the art performance on tagging Reddit data, as well as error
analysis of the results of these models, and offer a typology of the most
common error types among them, broken down by training corpus.
