  Transfer learning is an emerging paradigm for leveraging multiple sources to
improve the statistical inference on a single target. In this paper, we propose
a novel approach named residual importance weighted transfer learning (RIW-TL)
for high-dimensional linear models built on penalized likelihood. Compared to
existing methods such as Trans-Lasso that selects sources in an all-in-all-out
manner, RIW-TL includes samples via importance weighting and thus may permit
more effective sample use. To determine the weights, remarkably RIW-TL only
requires the knowledge of one-dimensional densities dependent on residuals,
thus overcoming the curse of dimensionality of having to estimate
high-dimensional densities in naive importance weighting. We show that the
oracle RIW-TL provides a faster rate than its competitors and develop a
cross-fitting procedure to estimate this oracle. We discuss variants of RIW-TL
by adopting different choices for residual weighting. The theoretical
properties of RIW-TL and its variants are established and compared with those
of LASSO and Trans-Lasso. Extensive simulation and a real data analysis confirm
its advantages.
