  Generalized text representations are the foundation of many natural language
understanding tasks. To fully utilize the different corpus, it is inevitable
that models need to understand the relevance among them. However, many methods
ignore the relevance and adopt a single-channel model (a coarse paradigm)
directly for all tasks, which lacks enough rationality and interpretation. In
addition, some existing works learn downstream tasks by stitches skill block(a
fine paradigm), which might cause irrationalresults due to its redundancy and
noise. Inthis work, we first analyze the task correlation through three
different perspectives, i.e., data property, manual design, and model-based
relevance, based on which the similar tasks are grouped together. Then, we
propose a hierarchical framework with a coarse-to-fine paradigm, with the
bottom level shared to all the tasks, the mid-level divided to different
groups, and the top-level assigned to each of the tasks. This allows our model
to learn basic language properties from all tasks, boost performance on
relevant tasks, and reduce the negative impact from irrelevant tasks. Our
experiments on 13 benchmark datasets across five natural language understanding
tasks demonstrate the superiority of our method.
