  Transfer learning is a useful technique for achieving improved performance
and reducing training costs by leveraging the knowledge gained from source
tasks and applying it to target tasks. Assessing the effectiveness of transfer
learning relies on understanding the similarity between the ground truth of the
source and target tasks. In real-world applications, tasks often exhibit
partial similarity, where certain aspects are similar while others are
different or irrelevant. To investigate the impact of partial similarity on
transfer learning performance, we focus on a linear regression model with two
distinct sets of features: a common part shared across tasks and a
task-specific part. Our study explores various types of transfer learning,
encompassing two options for parameter transfer. By establishing a theoretical
characterization on the error of the learned model, we compare these transfer
learning options, particularly examining how generalization performance changes
with the number of features/parameters in both underparameterized and
overparameterized regimes. Furthermore, we provide practical guidelines for
determining the number of features in the common and task-specific parts for
improved generalization performance. For example, when the total number of
features in the source task's learning model is fixed, we show that it is more
advantageous to allocate a greater number of redundant features to the
task-specific part rather than the common part. Moreover, in specific
scenarios, particularly those characterized by high noise levels and small true
parameters, sacrificing certain true features in the common part in favor of
employing more redundant features in the task-specific part can yield notable
benefits.
