  Large Language Models (LLMs) have transformed machine learning but raised
significant legal concerns due to their potential to produce text that
infringes on copyrights, resulting in several high-profile lawsuits. The legal
landscape is struggling to keep pace with these rapid advancements, with
ongoing debates about whether generated text might plagiarize copyrighted
materials. Current LLMs may infringe on copyrights or overly restrict
non-copyrighted texts, leading to these challenges: (i) the need for a
comprehensive evaluation benchmark to assess copyright compliance from multiple
aspects; (ii) evaluating robustness against safeguard bypassing attacks; and
(iii) developing effective defenses targeted against the generation of
copyrighted text. To tackle these challenges, we introduce a curated dataset to
evaluate methods, test attack strategies, and propose lightweight, real-time
defenses to prevent the generation of copyrighted text, ensuring the safe and
lawful use of LLMs. Our experiments demonstrate that current LLMs frequently
output copyrighted text, and that jailbreaking attacks can significantly
increase the volume of copyrighted output. Our proposed defense mechanisms
significantly reduce the volume of copyrighted text generated by LLMs by
effectively refusing malicious requests. Code is publicly available at
https://github.com/xz-liu/SHIELD
