  Federated learning allows distributed users to collaboratively train a model
while keeping each user's data private. Recently, a growing body of work has
demonstrated that an eavesdropping attacker can effectively recover image data
from gradients transmitted during federated learning. However, little progress
has been made in recovering text data. In this paper, we present a novel attack
method FILM for federated learning of language models (LMs). For the first
time, we show the feasibility of recovering text from large batch sizes of up
to 128 sentences. Unlike image-recovery methods that are optimized to match
gradients, we take a distinct approach that first identifies a set of words
from gradients and then directly reconstructs sentences based on beam search
and a prior-based reordering strategy. We conduct the FILM attack on several
large-scale datasets and show that it can successfully reconstruct single
sentences with high fidelity for large batch sizes and even multiple sentences
if applied iteratively. We evaluate three defense methods: gradient pruning,
DPSGD, and a simple approach to freeze word embeddings that we propose. We show
that both gradient pruning and DPSGD lead to a significant drop in utility.
However, if we fine-tune a public pre-trained LM on private text without
updating word embeddings, it can effectively defend the attack with minimal
data utility loss. Together, we hope that our results can encourage the
community to rethink the privacy concerns of LM training and its standard
practices in the future.
