  Recurrent Neural Network models are the state-of-the-art for Named Entity
Recognition (NER). We present two innovations to improve the performance of
these models. The first innovation is the introduction of residual connections
between the Stacked Recurrent Neural Network model to address the degradation
problem of deep neural networks. The second innovation is a bias decoding
mechanism that allows the trained system to adapt to non-differentiable and
externally computed objectives, such as the entity-based F-measure. Our work
improves the state-of-the-art results for both Spanish and English languages on
the standard train/development/test split of the CoNLL 2003 Shared Task NER
dataset.
