  We present a deep hierarchical recurrent neural network for sequence tagging.
Given a sequence of words, our model employs deep gated recurrent units on both
character and word levels to encode morphology and context information, and
applies a conditional random field layer to predict the tags. Our model is task
independent, language independent, and feature engineering free. We further
extend our model to multi-task and cross-lingual joint training by sharing the
architecture and parameters. Our model achieves state-of-the-art results in
multiple languages on several benchmark tasks including POS tagging, chunking,
and NER. We also demonstrate that multi-task and cross-lingual joint training
can improve the performance in various cases.
