  This paper proposes a novel, efficient transfer learning method, called
Scalable Weight Reparametrization (SWR) that is efficient and effective for
multiple downstream tasks. Efficient transfer learning involves utilizing a
pre-trained model trained on a larger dataset and repurposing it for downstream
tasks with the aim of maximizing the reuse of the pre-trained model. However,
previous works have led to an increase in updated parameters and task-specific
modules, resulting in more computations, especially for tiny models.
Additionally, there has been no practical consideration for controlling the
number of updated parameters. To address these issues, we suggest learning a
policy network that can decide where to reparametrize the pre-trained model,
while adhering to a given constraint for the number of updated parameters. The
policy network is only used during the transfer learning process and not
afterward. As a result, our approach attains state-of-the-art performance in a
proposed multi-lingual keyword spotting and a standard benchmark,
ImageNet-to-Sketch, while requiring zero additional computations and
significantly fewer additional parameters.
