  Pre-trained language models (PLMs) have demonstrated significant proficiency
in solving a wide range of general natural language processing (NLP) tasks.
Researchers have observed a direct correlation between the performance of these
models and their sizes. As a result, the sizes of these models have notably
expanded in recent years, persuading researchers to adopt the term large
language models (LLMs) to characterize the larger-sized PLMs. The size
expansion comes with a distinct capability called in-context learning (ICL),
which represents a special form of prompting and allows the models to be
utilized through the presentation of demonstration examples without
modifications to the model parameters. Although interesting, privacy concerns
have become a major obstacle in its widespread usage. Multiple studies have
examined the privacy risks linked to ICL and prompting in general, and have
devised techniques to alleviate these risks. Thus, there is a necessity to
organize these mitigation techniques for the benefit of the community. This
survey provides a systematic overview of the privacy protection methods
employed during ICL and prompting in general. We review, analyze, and compare
different methods under this paradigm. Furthermore, we provide a summary of the
resources accessible for the development of these frameworks. Finally, we
discuss the limitations of these frameworks and offer a detailed examination of
the promising areas that necessitate further exploration.
