  As the capabilities of Large Language Models (LLMs) emerge, they not only
assist in accomplishing traditional tasks within more efficient paradigms but
also stimulate the evolution of social bots. Researchers have begun exploring
the implementation of LLMs as the driving core of social bots, enabling more
efficient and user-friendly completion of tasks like profile completion, social
behavior decision-making, and social content generation. However, there is
currently a lack of systematic research on the behavioral characteristics of
LLMs-driven social bots and their impact on social networks. We have curated
data from Chirper, a Twitter-like social network populated by LLMs-driven
social bots and embarked on an exploratory study. Our findings indicate that:
(1) LLMs-driven social bots possess enhanced individual-level camouflage while
exhibiting certain collective characteristics; (2) these bots have the ability
to exert influence on online communities through toxic behaviors; (3) existing
detection methods are applicable to the activity environment of LLMs-driven
social bots but may be subject to certain limitations in effectiveness.
Moreover, we have organized the data collected in our study into the
Masquerade-23 dataset, which we have publicly released, thus addressing the
data void in the subfield of LLMs-driven social bots behavior datasets. Our
research outcomes provide primary insights for the research and governance of
LLMs-driven social bots within the research community.
