  Does the training of large language models potentially infringe upon code
licenses? Furthermore, are there any datasets available that can be safely used
for training these models without violating such licenses? In our study, we
assess the current trends in the field and the importance of incorporating code
into the training of large language models. Additionally, we examine publicly
available datasets to see whether these models can be trained on them without
the risk of legal issues in the future. To accomplish this, we compiled a list
of 53 large language models trained on file-level code. We then extracted their
datasets and analyzed how much they overlap with a dataset we created,
consisting exclusively of strong copyleft code.
  Our analysis revealed that every dataset we examined contained license
inconsistencies, despite being selected based on their associated repository
licenses. We analyzed a total of 514 million code files, discovering 38 million
exact duplicates present in our strong copyleft dataset. Additionally, we
examined 171 million file-leading comments, identifying 16 million with strong
copyleft licenses and another 11 million comments that discouraged copying
without explicitly mentioning a license. Based on the findings of our study,
which highlights the pervasive issue of license inconsistencies in large
language models trained on code, our recommendation for both researchers and
the community is to prioritize the development and adoption of best practices
for dataset creation and management.
