  Neural approaches to sequence labeling often use a Conditional Random Field
(CRF) to model their output dependencies, while Recurrent Neural Networks (RNN)
are used for the same purpose in other tasks. We set out to establish RNNs as
an attractive alternative to CRFs for sequence labeling. To do so, we address
one of the RNN's most prominent shortcomings, the fact that it is not exposed
to its own errors with the maximum-likelihood training. We frame the prediction
of the output sequence as a sequential decision-making process, where we train
the network with an adjusted actor-critic algorithm (AC-RNN). We
comprehensively compare this strategy with maximum-likelihood training for both
RNNs and CRFs on three structured-output tasks. The proposed AC-RNN efficiently
matches the performance of the CRF on NER and CCG tagging, and outperforms it
on Machine Transliteration. We also show that our training strategy is
significantly better than other techniques for addressing RNN's exposure bias,
such as Scheduled Sampling, and Self-Critical policy training.
