  Recent advances in Large Language Models (LLMs) have revolutionized code
generation, leading to widespread adoption of AI coding tools by developers.
However, LLMs can generate license-protected code without providing the
necessary license information, leading to potential intellectual property
violations during software production. This paper addresses the critical, yet
underexplored, issue of license compliance in LLM-generated code by
establishing a benchmark to evaluate the ability of LLMs to provide accurate
license information for their generated code. To establish this benchmark, we
conduct an empirical study to identify a reasonable standard for "striking
similarity" that excludes the possibility of independent creation, indicating a
copy relationship between the LLM output and certain open-source code. Based on
this standard, we propose an evaluation benchmark LiCoEval, to evaluate the
license compliance capabilities of LLMs. Using LiCoEval, we evaluate 14 popular
LLMs, finding that even top-performing LLMs produce a non-negligible proportion
(0.88% to 2.01%) of code strikingly similar to existing open-source
implementations. Notably, most LLMs fail to provide accurate license
information, particularly for code under copyleft licenses. These findings
underscore the urgent need to enhance LLM compliance capabilities in code
generation tasks. Our study provides a foundation for future research and
development to improve license compliance in AI-assisted software development,
contributing to both the protection of open-source software copyrights and the
mitigation of legal risks for LLM users.
