  Most tasks in NLP require labeled data. Data labeling is often done on
crowdsourcing platforms due to scalability reasons. However, publishing data on
public platforms can only be done if no privacy-relevant information is
included. Textual data often contains sensitive information like person names
or locations. In this work, we investigate how removing personally identifiable
information (PII) as well as applying differential privacy (DP) rewriting can
enable text with privacy-relevant information to be used for crowdsourcing. We
find that DP-rewriting before crowdsourcing can preserve privacy while still
leading to good label quality for certain tasks and data. PII-removal led to
good label quality in all examined tasks, however, there are no privacy
guarantees given.
