  Multi-task learning (MTL) has received considerable attention, and numerous
deep learning applications benefit from MTL with multiple objectives. However,
constructing multiple related tasks is difficult, and sometimes only a single
task is available for training in a dataset. To tackle this problem, we
explored the idea of using unsupervised clustering to construct a variety of
auxiliary tasks from unlabeled data or existing labeled data. We found that
some of these newly constructed tasks could exhibit semantic meanings
corresponding to certain human-specific attributes, but some were non-ideal. In
order to effectively reduce the impact of non-ideal auxiliary tasks on the main
task, we further proposed a novel meta-learning-based multi-task learning
approach, which trained the shared hidden layers on auxiliary tasks, while the
meta-optimization objective was to minimize the loss on the main task, ensuring
that the optimizing direction led to an improvement on the main task.
Experimental results across five image datasets demonstrated that the proposed
method significantly outperformed existing single task learning,
semi-supervised learning, and some data augmentation methods, including an
improvement of more than 9% on the Omniglot dataset.
