  As large language models (LLMs) become increasingly commonplace, concern
about distinguishing between human and AI text increases as well. The growing
power of these models is of particular concern to teachers, who may worry that
students will use LLMs to write school assignments. Facing a technology with
which they are unfamiliar, teachers may turn to publicly-available AI text
detectors. Yet the accuracy of many of these detectors has not been thoroughly
verified, posing potential harm to students who are falsely accused of academic
dishonesty. In this paper, we evaluate three different AI text
detectors-Kirchenbauer et al. watermarks, ZeroGPT, and GPTZero-against human
and AI-generated essays. We find that watermarking results in a high false
positive rate, and that ZeroGPT has both high false positive and false negative
rates. Further, we are able to significantly increase the false negative rate
of all detectors by using ChatGPT 3.5 to paraphrase the original AI-generated
texts, thereby effectively bypassing the detectors.
