  Prompt-tuning has received attention as an efficient tuning method in the
language domain, i.e., tuning a prompt that is a few tokens long, while keeping
the large language model frozen, yet achieving comparable performance with
conventional fine-tuning. Considering the emerging privacy concerns with
language models, we initiate the study of privacy leakage in the setting of
prompt-tuning. We first describe a real-world email service pipeline to provide
customized output for various users via prompt-tuning. Then we propose a novel
privacy attack framework to infer users' private information by exploiting the
prompt module with user-specific signals. We conduct a comprehensive privacy
evaluation on the target pipeline to demonstrate the potential leakage from
prompt-tuning. The results also demonstrate the effectiveness of the proposed
attack.
