  Training neural machine translation (NMT) models in federated learning (FL)
settings could be inefficient both computationally and communication-wise, due
to the large size of translation engines as well as the multiple rounds of
updates required to train clients and a central server. In this paper, we
explore how to efficiently build NMT models in an FL setup by proposing a novel
solution. In order to reduce the communication overhead, out of all neural
layers we only exchange what we term "Controller" layers. Controllers are a
small number of additional neural components connected to our pre-trained
architectures. These new components are placed in between original layers. They
act as liaisons to communicate with the central server and learn minimal
information that is sufficient enough to update clients.
  We evaluated the performance of our models on five datasets from different
domains to translate from German into English. We noted that the models
equipped with Controllers preform on par with those trained in a central and
non-FL setting. In addition, we observed a substantial reduction in the
communication traffic of the FL pipeline, which is a direct consequence of
using Controllers. Based on our experiments, Controller-based models are ~6
times less expensive than their other peers. This reduction is significantly
important when we consider the number of parameters in large models and it
becomes even more critical when such parameters need to be exchanged for
multiple rounds in FL settings.
