  Distributed representation plays an important role in deep learning based
natural language processing. However, the representation of a sentence often
varies in different tasks, which is usually learned from scratch and suffers
from the limited amounts of training data. In this paper, we claim that a good
sentence representation should be invariant and can benefit the various
subsequent tasks. To achieve this purpose, we propose a new scheme of
information sharing for multi-task learning. More specifically, all tasks share
the same sentence representation and each task can select the task-specific
information from the shared sentence representation with attention mechanism.
The query vector of each task's attention could be either static parameters or
generated dynamically. We conduct extensive experiments on 16 different text
classification tasks, which demonstrate the benefits of our architecture.
