  With the recent proliferation of Large Language Models (LLMs), there has been
an increasing demand for tools to detect machine-generated text. The effective
detection of machine-generated text face two pertinent problems: First, they
are severely limited in generalizing against real-world scenarios, where
machine-generated text is produced by a variety of generators, including but
not limited to GPT-4 and Dolly, and spans diverse domains, ranging from
academic manuscripts to social media posts. Second, existing detection
methodologies treat texts produced by LLMs through a restrictive binary
classification lens, neglecting the nuanced diversity of artifacts generated by
different LLMs. In this work, we undertake a systematic study on the detection
of machine-generated text in real-world scenarios. We first study the
effectiveness of state-of-the-art approaches and find that they are severely
limited against text produced by diverse generators and domains in the real
world. Furthermore, t-SNE visualizations of the embeddings from a pretrained
LLM's encoder show that they cannot reliably distinguish between human and
machine-generated text. Based on our findings, we introduce a novel system,
T5LLMCipher, for detecting machine-generated text using a pretrained T5 encoder
combined with LLM embedding sub-clustering to address the text produced by
diverse generators and domains in the real world. We evaluate our approach
across 9 machine-generated text systems and 9 domains and find that our
approach provides state-of-the-art generalization ability, with an average
increase in F1 score on machine-generated text of 19.6\% on unseen generators
and domains compared to the top performing existing approaches and correctly
attributes the generator of text with an accuracy of 93.6\%.
