  LLMs offer valuable capabilities, yet they can be utilized by malicious users
to disseminate deceptive information and generate fake news. The growing
prevalence of LLMs poses difficulties in crafting detection approaches that
remain effective across various text domains. Additionally, the absence of
precautionary measures for AI-generated news on online social platforms is
concerning. Therefore, there is an urgent need to improve people's ability to
differentiate between news articles written by humans and those produced by
LLMs. By providing cues in human-written and LLM-generated news, we can help
individuals increase their skepticism towards fake LLM-generated news. This
paper aims to elucidate simple markers that help individuals distinguish
between articles penned by humans and those created by LLMs. To achieve this,
we initially collected a dataset comprising 39k news articles authored by
humans or generated by four distinct LLMs with varying degrees of fake. We then
devise a metric named Entropy-Shift Authorship Signature (ESAS) based on the
information theory and entropy principles. The proposed ESAS ranks terms or
entities, like POS tagging, within news articles based on their relevance in
discerning article authorship. We demonstrate the effectiveness of our metric
by showing the high accuracy attained by a basic method, i.e., TF-IDF combined
with logistic regression classifier, using a small set of terms with the
highest ESAS score. Consequently, we introduce and scrutinize these top
ESAS-ranked terms to aid individuals in strengthening their skepticism towards
LLM-generated fake news.
