  Deep learning approaches have achieved great success in the field of Natural
Language Processing (NLP). However, directly training deep neural models often
suffer from overfitting and data scarcity problems that are pervasive in NLP
tasks. In recent years, Multi-Task Learning (MTL), which can leverage useful
information of related tasks to achieve simultaneous performance improvement on
these tasks, has been used to handle these problems. In this paper, we give an
overview of the use of MTL in NLP tasks. We first review MTL architectures used
in NLP tasks and categorize them into four classes, including parallel
architecture, hierarchical architecture, modular architecture, and generative
adversarial architecture. Then we present optimization techniques on loss
construction, gradient regularization, data sampling, and task scheduling to
properly train a multi-task model. After presenting applications of MTL in a
variety of NLP tasks, we introduce some benchmark datasets. Finally, we make a
conclusion and discuss several possible research directions in this field.
