  A multi-task learning (MTL) system aims at solving multiple related tasks at
the same time. With a fixed model capacity, the tasks would be conflicted with
each other, and the system usually has to make a trade-off among learning all
of them together. For many real-world applications where the trade-off has to
be made online, multiple models with different preferences over tasks have to
be trained and stored. This work proposes a novel controllable Pareto
multi-task learning framework, to enable the system to make real-time trade-off
control among different tasks with a single model. To be specific, we formulate
the MTL as a preference-conditioned multiobjective optimization problem, with a
parametric mapping from preferences to the corresponding trade-off solutions. A
single hypernetwork-based multi-task neural network is built to learn all tasks
with different trade-off preferences among them, where the hypernetwork
generates the model parameters conditioned on the preference. For inference,
MTL practitioners can easily control the model performance based on different
trade-off preferences in real-time. Experiments on different applications
demonstrate that the proposed model is efficient for solving various MTL
problems.
