  In this paper, we extend a class of celebrated regularization techniques
originally proposed for feed-forward neural networks, namely Input Mixup (Zhang
et al., 2017) and Manifold Mixup (Verma et al., 2018), to the realm of
Recurrent Neural Networks (RNN). Our proposed methods are easy to implement and
have a low computational complexity, while leverage the performance of simple
neural architectures in a variety of tasks. We have validated our claims
through several experiments on real-world datasets, and also provide an
asymptotic theoretical analysis to further investigate the properties and
potential impacts of our proposed techniques. Applying sequence mixup to
BiLSTM-CRF model (Huang et al., 2015) to Named Entity Recognition task on
CoNLL-2003 data (Sang and De Meulder, 2003) has improved the F-1 score on the
test stage and reduced the loss, considerably.
