  A key factor in developing high performing machine learning models is the
availability of sufficiently large datasets. This work is motivated by
applications arising in Software as a Service (SaaS) companies where there
exist numerous similar yet disjoint datasets from multiple client companies. To
overcome the challenges of insufficient data without explicitly aggregating the
clients' datasets due to privacy concerns, one solution is to collect more data
for each individual client, another is to privately aggregate information from
models trained on each client's data. In this work, two approaches for private
model aggregation are proposed that enable the transfer of knowledge from
existing models trained on other companies' datasets to a new company with
limited labeled data while protecting each client company's underlying
individual sensitive information. The two proposed approaches are based on
state-of-the-art private learning algorithms: Differentially Private
Permutation-based Stochastic Gradient Descent and Approximate Minima
Perturbation. We empirically show that by leveraging differentially private
techniques, we can enable private model aggregation and augment data utility
while providing provable mathematical guarantees on privacy. The proposed
methods thus provide significant business value for SaaS companies and their
clients, specifically as a solution for the cold-start problem.
