  In this paper we present a novel Neural Network algorithm for conducting
semi-supervised learning for sequence labeling tasks arranged in a
linguistically motivated hierarchy. This relationship is exploited to
regularise the representations of supervised tasks by backpropagating the error
of the unsupervised task through the supervised tasks. We introduce a neural
network where lower layers are supervised by junior downstream tasks and the
final layer task is an auxiliary unsupervised task. The architecture shows
improvements of up to two percentage points F1 for Chunking compared to a
plausible baseline.
