  Multi-task learning (MTL) aims to make full use of the knowledge contained in
multi-task supervision signals to improve the overall performance. How to make
the knowledge of multiple tasks shared appropriately is an open problem for
MTL. Most existing deep MTL models are based on parameter sharing. However,
suitable sharing mechanism is hard to design as the relationship among tasks is
complicated. In this paper, we propose a general framework called Multi-Task
Neural Architecture Search (MTNAS) to efficiently find a suitable sharing route
for a given MTL problem. MTNAS modularizes the sharing part into multiple
layers of sub-networks. It allows sparse connection among these sub-networks
and soft sharing based on gating is enabled for a certain route. Benefiting
from such setting, each candidate architecture in our search space defines a
dynamic sparse sharing route which is more flexible compared with full-sharing
in previous approaches. We show that existing typical sharing approaches are
sub-graphs in our search space. Extensive experiments on three real-world
recommendation datasets demonstrate MTANS achieves consistent improvement
compared with single-task models and typical multi-task methods while
maintaining high computation efficiency. Furthermore, in-depth experiments
demonstrates that MTNAS can learn suitable sparse route to mitigate negative
transfer.
