  Tree-structured multi-task architectures have been employed to jointly tackle
multiple vision tasks in the context of multi-task learning (MTL). The major
challenge is to determine where to branch out for each task given a backbone
model to optimize for both task accuracy and computation efficiency. To address
the challenge, this paper proposes a recommender that, given a set of tasks and
a convolutional neural network-based backbone model, automatically suggests
tree-structured multi-task architectures that could achieve a high task
performance while meeting a user-specified computation budget without
performing model training. Extensive evaluations on popular MTL benchmarks show
that the recommended architectures could achieve competitive task accuracy and
computation efficiency compared with state-of-the-art MTL methods. Our
tree-structured multi-task model recommender is open-sourced and available at
https://github.com/zhanglijun95/TreeMTL.
