  To combat the misuse of Large Language Models (LLMs), many recent studies
have presented LLM-generated-text detectors with promising performance. When
users instruct LLMs to generate texts, the instruction can include different
constraints depending on the user's need. However, most recent studies do not
cover such diverse instruction patterns when creating datasets for LLM
detection. In this paper, we reveal that even task-oriented constraints --
constraints that would naturally be included in an instruction and are not
related to detection-evasion -- cause existing powerful detectors to have a
large variance in detection performance. We focus on student essay writing as a
realistic domain and manually create task-oriented constraints based on several
factors for essay quality. Our experiments show that the standard deviation
(SD) of current detector performance on texts generated by an instruction with
such a constraint is significantly larger (up to an SD of 14.4 F1-score) than
that by generating texts multiple times or paraphrasing the instruction. We
also observe an overall trend where the constraints can make LLM detection more
challenging than without them. Finally, our analysis indicates that the high
instruction-following ability of LLMs fosters the large impact of such
constraints on detection performance.
