  Collaborative training can improve the accuracy of a model for a user by
trading off the model's bias (introduced by using data from other users who are
potentially different) against its variance (due to the limited amount of data
on any single user). In this work, we formalize the personalized collaborative
learning problem as a stochastic optimization of a task 0 while giving access
to N related but different tasks 1,..., N. We provide convergence guarantees
for two algorithms in this setting -- a popular collaboration method known as
weighted gradient averaging, and a novel bias correction method -- and explore
conditions under which we can achieve linear speedup w.r.t. the number of
auxiliary tasks N. Further, we also empirically study their performance
confirming our theoretical insights.
