  Many real-world machine learning applications involve several learning tasks
which are inter-related. For example, in healthcare domain, we need to learn a
predictive model of a certain disease for many hospitals. The models for each
hospital may be different because of the inherent differences in the
distributions of the patient populations. However, the models are also closely
related because of the nature of the learning tasks modeling the same disease.
By simultaneously learning all the tasks, multi-task learning (MTL) paradigm
performs inductive knowledge transfer among tasks to improve the generalization
performance. When datasets for the learning tasks are stored at different
locations, it may not always be feasible to transfer the data to provide a
data-centralized computing environment due to various practical issues such as
high data volume and privacy. In this paper, we propose a principled MTL
framework for distributed and asynchronous optimization to address the
aforementioned challenges. In our framework, gradient update does not wait for
collecting the gradient information from all the tasks. Therefore, the proposed
method is very efficient when the communication delay is too high for some task
nodes. We show that many regularized MTL formulations can benefit from this
framework, including the low-rank MTL for shared subspace learning. Empirical
studies on both synthetic and real-world datasets demonstrate the efficiency
and effectiveness of the proposed framework.
