  A central tenet of Federated learning (FL), which trains models without
centralizing user data, is privacy. However, previous work has shown that the
gradient updates used in FL can leak user information. While the most
industrial uses of FL are for text applications (e.g. keystroke prediction),
nearly all attacks on FL privacy have focused on simple image classifiers. We
propose a novel attack that reveals private user text by deploying malicious
parameter vectors, and which succeeds even with mini-batches, multiple users,
and long sequences. Unlike previous attacks on FL, the attack exploits
characteristics of both the Transformer architecture and the token embedding,
separately extracting tokens and positional embeddings to retrieve
high-fidelity text. This work suggests that FL on text, which has historically
been resistant to privacy attacks, is far more vulnerable than previously
thought.
