  Lifelong machine learning methods acquire knowledge over a series of
consecutive tasks, continually building upon their experience. Current lifelong
learning algorithms rely upon a single learning agent that has centralized
access to all data. In this paper, we extend the idea of lifelong learning from
a single agent to a network of multiple agents that collectively learn a series
of tasks. Each agent faces some (potentially unique) set of tasks; the key idea
is that knowledge learned from these tasks may benefit other agents trying to
learn different (but related) tasks. Our Collective Lifelong Learning Algorithm
(CoLLA) provides an efficient way for a network of agents to share their
learned knowledge in a distributed and decentralized manner, while preserving
the privacy of the locally observed data. Note that a decentralized scheme is a
subclass of distributed algorithms where a central server does not exist and in
addition to data, computations are also distributed among the agents. We
provide theoretical guarantees for robust performance of the algorithm and
empirically demonstrate that CoLLA outperforms existing approaches for
distributed multi-task learning on a variety of data sets.
