Multi-Task Learning in Natural Language Processing: An Overview
  Deep learning approaches have achieved great success in the field of Natural
Language Processing (NLP). However, deep neural models often suffer from
overfitting and data scarcity problems that are pervasive in NLP tasks. In
recent years, Multi-Task Learning (MTL), which can leverage useful information
of related tasks to achieve simultaneous performance improvement on multiple
related tasks, has been used to handle these problems. In this paper, we give
an overview of the use of MTL in NLP tasks. We first review MTL architectures
used in NLP tasks and categorize them into four classes, including the parallel
architecture, hierarchical architecture, modular architecture, and generative
adversarial architecture. Then we present optimization techniques on loss
construction, data sampling, and task scheduling to properly train a multi-task
model. After presenting applications of MTL in a variety of NLP tasks, we
introduce some benchmark datasets. Finally, we make a conclusion and discuss
several possible research directions in this field.
