  The recent rise of generative artificial intelligence (GenAI) capable of
creating scientific images presents a challenge in the fight against academic
fraud. This study evaluates the efficacy of three free web-based AI detectors
in identifying AI-generated images of Western blots, which is a very common
technique in biology. We tested these detectors on a collection of artificial
Western blot images (n=48) that were created using ChatGPT 4 DALLE 3 and on
authentic Western blots (n=48) that were sampled from articles published within
four biology journals in 2015; this was before the rise of generative AI based
on large language models. The results reveal that the sensitivity (0.9583 for
Is It AI, 0.1875 for Hive Moderation, and 0.7083 for Illuminarty) and
specificity (0.5417 for Is It AI, 0.8750 for Hive Moderation, and 0.4167 for
Illuminarty) are very different. Positive predictive values (PPV) across
various AI prevalence were low, for example reaching 0.1885 for Is It AI,
0.1429 for Hive Moderation, and 0.1189 for Illuminarty at an AI prevalence of
0.1. This highlights the difficulty in confidently determining image
authenticity based on the output of a single detector. Reducing the size of
Western blots from four to two lanes reduced test sensitivities and increased
test specificities but did not markedly affect overall detector accuracies and
also only slightly improved the PPV of one detector (Is It AI). These findings
strongly argue against the use of free AI detectors to detect fake scientific
images, and they demonstrate the urgent need for more robust detection tools
that are specifically trained on scientific content such as Western blot
images.
