  With the widespread adoption of Large Language Models (LLMs), concerns about
potential misuse have emerged. To this end, watermarking has been adapted to
LLM, enabling a simple and effective way to detect and monitor generated text.
However, while the existing methods can differentiate between watermarked and
unwatermarked text with high accuracy, they often face a trade-off between the
quality of the generated text and the effectiveness of the watermarking
process. In this work, we present a novel type of LLM watermark, Sparse
Watermark, which aims to mitigate this trade-off by applying watermarks to a
small subset of generated tokens distributed across the text. The key strategy
involves anchoring watermarked tokens to words that have specific
Part-of-Speech (POS) tags. Our experimental results demonstrate that the
proposed watermarking scheme achieves high detectability while generating text
that outperforms previous LLM watermarking methods in quality across various
tasks
