  As deep learning applications continue to become more diverse, an interesting
question arises: Can general problem solving arise from jointly learning
several such diverse tasks? To approach this question, deep multi-task learning
is extended in this paper to the setting where there is no obvious overlap
between task architectures. The idea is that any set of (architecture,task)
pairs can be decomposed into a set of potentially related subproblems, whose
sharing is optimized by an efficient stochastic algorithm. The approach is
first validated in a classic synthetic multi-task learning benchmark, and then
applied to sharing across disparate architectures for vision, NLP, and genomics
tasks. It discovers regularities across these domains, encodes them into
sharable modules, and combines these modules systematically to improve
performance in the individual tasks. The results confirm that sharing learned
functionality across diverse domains and architectures is indeed beneficial,
thus establishing a key ingredient for general problem solving in the future.
