  We propose a novel adaptive transfer learning framework, learning to transfer
learn (L2TL), to improve performance on a target dataset by careful extraction
of the related information from a source dataset. Our framework considers
cooperative optimization of shared weights between models for source and target
tasks, and adjusts the constituent loss weights adaptively. The adaptation of
the weights is based on a reinforcement learning (RL) selection policy, guided
with a performance metric on the target validation set. We demonstrate that
L2TL outperforms fine-tuning baselines and other adaptive transfer learning
methods on eight datasets. In the regimes of small-scale target datasets and
significant label mismatch between source and target datasets, L2TL shows
particularly large benefits.
