  This paper delves into the intricacies of code summarization using advanced
transformer-based language models. Through empirical studies, we evaluate the
efficacy of code summarization by altering function and variable names to
explore whether models truly understand code semantics or merely rely on
textual cues. We have also introduced adversaries like dead code and commented
code across three programming languages (Python, Javascript, and Java) to
further scrutinize the model's understanding. Ultimately, our research aims to
offer valuable insights into the inner workings of transformer-based LMs,
enhancing their ability to understand code and contributing to more efficient
software development practices and maintenance workflows.
