  Privatized text rewriting with local differential privacy (LDP) is a recent
approach that enables sharing of sensitive textual documents while formally
guaranteeing privacy protection to individuals. However, existing systems face
several issues, such as formal mathematical flaws, unrealistic privacy
guarantees, privatization of only individual words, as well as a lack of
transparency and reproducibility. In this paper, we propose a new system
'DP-BART' that largely outperforms existing LDP systems. Our approach uses a
novel clipping method, iterative pruning, and further training of internal
representations which drastically reduces the amount of noise required for DP
guarantees. We run experiments on five textual datasets of varying sizes,
rewriting them at different privacy guarantees and evaluating the rewritten
texts on downstream text classification tasks. Finally, we thoroughly discuss
the privatized text rewriting approach and its limitations, including the
problem of the strict text adjacency constraint in the LDP paradigm that leads
to the high noise requirement.
