  Parameter-transfer is a well-known and versatile approach for meta-learning,
with applications including few-shot learning, federated learning, and
reinforcement learning. However, parameter-transfer algorithms often require
sharing models that have been trained on the samples from specific tasks, thus
leaving the task-owners susceptible to breaches of privacy. We conduct the
first formal study of privacy in this setting and formalize the notion of
task-global differential privacy as a practical relaxation of more commonly
studied threat models. We then propose a new differentially private algorithm
for gradient-based parameter transfer that not only satisfies this privacy
requirement but also retains provable transfer learning guarantees in convex
settings. Empirically, we apply our analysis to the problems of federated
learning with personalization and few-shot classification, showing that
allowing the relaxation to task-global privacy from the more commonly studied
notion of local privacy leads to dramatically increased performance in
recurrent neural language modeling and image classification.
