  Large language models (LLMs) encapsulate vast amounts of knowledge but still
remain vulnerable to external misinformation. Existing research mainly studied
this susceptibility behavior in a single-turn setting. However, belief can
change during a multi-turn conversation, especially a persuasive one.
Therefore, in this study, we delve into LLMs' susceptibility to persuasive
conversations, particularly on factual questions that they can answer
correctly. We first curate the Farm (i.e., Fact to Misinform) dataset, which
contains factual questions paired with systematically generated persuasive
misinformation. Then, we develop a testing framework to track LLMs' belief
changes in a persuasive dialogue. Through extensive experiments, we find that
LLMs' correct beliefs on factual knowledge can be easily manipulated by various
persuasive strategies.
