  Modern deep learning systems are increasingly deployed in situations such as
personalization and federated learning where it is necessary to support i)
learning on small amounts of data, and ii) communication efficient distributed
training protocols. In this work, we develop FiLM Transfer (FiT) which fulfills
these requirements in the image classification setting by combining ideas from
transfer learning (fixed pretrained backbones and fine-tuned FiLM adapter
layers) and meta-learning (automatically configured Naive Bayes classifiers and
episodic training) to yield parameter efficient models with superior
classification accuracy at low-shot. The resulting parameter efficiency is key
for enabling few-shot learning, inexpensive model updates for personalization,
and communication efficient federated learning. We experiment with FiT on a
wide range of downstream datasets and show that it achieves better
classification accuracy than the leading Big Transfer (BiT) algorithm at
low-shot and achieves state-of-the art accuracy on the challenging VTAB-1k
benchmark, with fewer than 1% of the updateable parameters. Finally, we
demonstrate the parameter efficiency and superior accuracy of FiT in
distributed low-shot applications including model personalization and federated
learning where model update size is an important performance metric.
