  In many practical applications of supervised learning the task involves the
prediction of multiple target variables from a common set of input variables.
When the prediction targets are binary the task is called multi-label
classification, while when the targets are continuous the task is called
multi-target regression. In both tasks, target variables often exhibit
statistical dependencies and exploiting them in order to improve predictive
accuracy is a core challenge. A family of multi-label classification methods
address this challenge by building a separate model for each target on an
expanded input space where other targets are treated as additional input
variables. Despite the success of these methods in the multi-label
classification domain, their applicability and effectiveness in multi-target
regression has not been studied until now. In this paper, we introduce two new
methods for multi-target regression, called Stacked Single-Target and Ensemble
of Regressor Chains, by adapting two popular multi-label classification methods
of this family. Furthermore, we highlight an inherent problem of these methods
- a discrepancy of the values of the additional input variables between
training and prediction - and develop extensions that use out-of-sample
estimates of the target variables during training in order to tackle this
problem. The results of an extensive experimental evaluation carried out on a
large and diverse collection of datasets show that, when the discrepancy is
appropriately mitigated, the proposed methods attain consistent improvements
over the independent regressions baseline. Moreover, two versions of Ensemble
of Regression Chains perform significantly better than four state-of-the-art
methods including regularization-based multi-task learning methods and a
multi-objective random forest approach.
