  Online users generate tremendous amounts of textual information by
participating in different activities, such as writing reviews and sharing
tweets. This textual data provides opportunities for researchers and business
partners to study and understand individuals. However, this user-generated
textual data not only can reveal the identity of the user but also may contain
individual's private information (e.g., age, location, gender). Hence, "you are
what you write" as the saying goes. Publishing the textual data thus
compromises the privacy of individuals who provided it. The need arises for
data publishers to protect people's privacy by anonymizing the data before
publishing it. It is challenging to design effective anonymization techniques
for textual information which minimizes the chances of re-identification and
does not contain users' sensitive information (high privacy) while retaining
the semantic meaning of the data for given tasks (high utility). In this paper,
we study this problem and propose a novel double privacy preserving text
representation learning framework, DPText, which learns a textual
representation that (1) is differentially private, (2) does not contain private
information and (3) retains high utility for the given task. Evaluating on two
natural language processing tasks, i.e., sentiment analysis and part of speech
tagging, we show the effectiveness of this approach in terms of preserving both
privacy and utility.
