  Learning two tasks in a single shared function has some benefits. Firstly by
acquiring information from the second task, the shared function leverages
useful information that could have been neglected or underestimated in the
first task. Secondly, it helps to generalize the function that can be learned
using generally applicable information for both tasks. To fully enjoy these
benefits, Multi-task Learning (MTL) has long been researched in various domains
such as computer vision, language understanding, and speech synthesis. While
MTL benefits from the positive transfer of information from multiple tasks, in
a real environment, tasks inevitably have a conflict between them during the
learning phase, called negative transfer. The negative transfer hampers
function from achieving the optimality and degrades the performance. To solve
the problem of the task conflict, previous works only suggested partial
solutions that are not fundamental, but ad-hoc. A common approach is using a
weighted sum of losses. The weights are adjusted to induce positive transfer.
Paradoxically, this kind of solution acknowledges the problem of negative
transfer and cannot remove it unless the weight of the task is set to zero.
Therefore, these previous methods had limited success. In this paper, we
introduce a novel approach that can drive positive transfer and suppress
negative transfer by leveraging class-wise weights in the learning process. The
weights act as an arbitrator of the fundamental unit of information to
determine its positive or negative status to the main task.
