  Learning-based techniques, especially advanced Large Language Models (LLMs)
for code, have gained considerable popularity in various software engineering
(SE) tasks. However, most existing works focus on designing better
learning-based models and pay less attention to the properties of datasets.
Learning-based models, including popular LLMs for code, heavily rely on data,
and the data's properties (e.g., data distribution) could significantly affect
their behavior. We conducted an exploratory study on the distribution of SE
data and found that such data usually follows a skewed distribution (i.e.,
long-tailed distribution) where a small number of classes have an extensive
collection of samples, while a large number of classes have very few samples.
We investigate three distinct SE tasks and analyze the impacts of long-tailed
distribution on the performance of LLMs for code. Our experimental results
reveal that the long-tailed distribution has a substantial impact on the
effectiveness of LLMs for code. Specifically, LLMs for code perform between
30.0\% and 254.0\% worse on data samples associated with infrequent labels
compared to data samples of frequent labels. Our study provides a better
understanding of the effects of long-tailed distributions on popular LLMs for
code and insights for the future development of SE automation.
