  AI-Generated Content (AIGC) is gaining great popularity, with many emerging
commercial services and applications. These services leverage advanced
generative models, such as latent diffusion models and large language models,
to generate creative content (e.g., realistic images and fluent sentences) for
users. The usage of such generated content needs to be highly regulated, as the
service providers need to ensure the users do not violate the usage policies
(e.g., abuse for commercialization, generating and distributing unsafe
content). A promising solution to achieve this goal is watermarking, which adds
unique and imperceptible watermarks on the content for service verification and
attribution. Numerous watermarking approaches have been proposed recently.
However, in this paper, we show that an adversary can easily break these
watermarking mechanisms. Specifically, we consider two possible attacks. (1)
Watermark removal: the adversary can easily erase the embedded watermark from
the generated content and then use it freely bypassing the regulation of the
service provider. (2) Watermark forging: the adversary can create illegal
content with forged watermarks from another user, causing the service provider
to make wrong attributions. We propose Warfare, a unified methodology to
achieve both attacks in a holistic way. The key idea is to leverage a
pre-trained diffusion model for content processing and a generative adversarial
network for watermark removal or forging. We evaluate Warfare on different
datasets and embedding setups. The results prove that it can achieve high
success rates while maintaining the quality of the generated content. Compared
to existing diffusion model-based attacks, Warfare is 5,050~11,000x faster.
