  This paper addresses the ethical concerns arising from the use of
unauthorized public data in deep learning models and proposes a novel solution.
Specifically, building on the work of Huang et al. (2021), we extend their
bi-level optimization approach to generate unlearnable text using a
gradient-based search technique. However, although effective, this approach
faces practical limitations, including the requirement of batches of instances
and model architecture knowledge that is not readily accessible to ordinary
users with limited access to their own data. Furthermore, even with
semantic-preserving constraints, unlearnable noise can alter the text's
semantics. To address these challenges, we extract simple patterns from
unlearnable text produced by bi-level optimization and demonstrate that the
data remains unlearnable for unknown models. Additionally, these patterns are
not instance- or dataset-specific, allowing users to readily apply them to text
classification and question-answering tasks, even if only a small proportion of
users implement them on their public content. We also open-source codes to
generate unlearnable text and assess unlearnable noise to benefit the public
and future studies.
