  With the development of new sensors and monitoring devices, more sources of
data become available to be used as inputs for machine learning models. These
can on the one hand help to improve the accuracy of a model. On the other hand,
combining these new inputs with historical data remains a challenge that has
not yet been studied in enough detail. In this work, we propose a transfer
learning algorithm that combines new and historical data with different input
dimensions. This approach is easy to implement, efficient, with computational
complexity equivalent to the ordinary least-squares method, and requires no
hyperparameter tuning, making it straightforward to apply when the new data is
limited. Different from other approaches, we provide a rigorous theoretical
study of its robustness, showing that it cannot be outperformed by a baseline
that utilizes only the new data. Our approach achieves state-of-the-art
performance on 9 real-life datasets, outperforming the linear DSFT, another
linear transfer learning algorithm, and performing comparably to non-linear
DSFT.
