  Multi-task learning is to improve the performance of the model by
transferring and exploiting common knowledge among tasks. Existing MTL works
mainly focus on the scenario where label sets among multiple tasks (MTs) are
usually the same, thus they can be utilized for learning across the tasks.
While almost rare works explore the scenario where each task only has a small
amount of training samples, and their label sets are just partially overlapped
or even not. Learning such MTs is more challenging because of less correlation
information available among these tasks. For this, we propose a framework to
learn these tasks by jointly leveraging both abundant information from a learnt
auxiliary big task with sufficiently many classes to cover those of all these
tasks and the information shared among those partially-overlapped tasks. In our
implementation of using the same neural network architecture of the learnt
auxiliary task to learn individual tasks, the key idea is to utilize available
label information to adaptively prune the hidden layer neurons of the auxiliary
network to construct corresponding network for each task, while accompanying a
joint learning across individual tasks. Our experimental results demonstrate
its effectiveness in comparison with the state-of-the-art approaches.
