  In this paper, we investigate a novel deep-model reusing task. Our goal is to
train a lightweight and versatile student model, without human-labelled
annotations, that amalgamates the knowledge and masters the expertise of two
pretrained teacher models working on heterogeneous problems, one on scene
parsing and the other on depth estimation. To this end, we propose an
innovative training strategy that learns the parameters of the student
intertwined with the teachers, achieved by 'projecting' its amalgamated
features onto each teacher's domain and computing the loss. We also introduce
two options to generalize the proposed training strategy to handle three or
more tasks simultaneously. The proposed scheme yields very encouraging results.
As demonstrated on several benchmarks, the trained student model achieves
results even superior to those of the teachers in their own expertise domains
and on par with the state-of-the-art fully supervised models relying on
human-labelled annotations.
