  Recent generative AI systems have demonstrated more advanced persuasive
capabilities and are increasingly permeating areas of life where they can
influence decision-making. Generative AI presents a new risk profile of
persuasion due the opportunity for reciprocal exchange and prolonged
interactions. This has led to growing concerns about harms from AI persuasion
and how they can be mitigated, highlighting the need for a systematic study of
AI persuasion. The current definitions of AI persuasion are unclear and related
harms are insufficiently studied. Existing harm mitigation approaches
prioritise harms from the outcome of persuasion over harms from the process of
persuasion. In this paper, we lay the groundwork for the systematic study of AI
persuasion. We first put forward definitions of persuasive generative AI. We
distinguish between rationally persuasive generative AI, which relies on
providing relevant facts, sound reasoning, or other forms of trustworthy
evidence, and manipulative generative AI, which relies on taking advantage of
cognitive biases and heuristics or misrepresenting information. We also put
forward a map of harms from AI persuasion, including definitions and examples
of economic, physical, environmental, psychological, sociocultural, political,
privacy, and autonomy harm. We then introduce a map of mechanisms that
contribute to harmful persuasion. Lastly, we provide an overview of approaches
that can be used to mitigate against process harms of persuasion, including
prompt engineering for manipulation classification and red teaming. Future work
will operationalise these mitigations and study the interaction between
different types of mechanisms of persuasion.
