  Large scale adoption of large language models has introduced a new era of
convenient knowledge transfer for a slew of natural language processing tasks.
However, these models also run the risk of undermining user trust by exposing
unwanted information about the data subjects, which may be extracted by a
malicious party, e.g. through adversarial attacks. We present an empirical
investigation into the extent of the personal information encoded into
pre-trained representations by a range of popular models, and we show a
positive correlation between the complexity of a model, the amount of data used
in pre-training, and data leakage. In this paper, we present the first wide
coverage evaluation and comparison of some of the most popular
privacy-preserving algorithms, on a large, multi-lingual dataset on sentiment
analysis annotated with demographic information (location, age and gender). The
results show since larger and more complex models are more prone to leaking
private information, use of privacy-preserving methods is highly desirable. We
also find that highly privacy-preserving technologies like differential privacy
(DP) can have serious model utility effects, which can be ameliorated using
hybrid or metric-DP techniques.
