  Pre-trained Large Language Models (LLM) have achieved remarkable successes in
several domains. However, code-oriented LLMs are heavy in computational
complexity, and quadratically with the length of the input. Toward simplifying
the input program of an LLM, the state-of-the-art approach has the strategies
to filter the input code tokens based on the attention scores given by the LLM.
The decision to simplify the input should not rely on the attention patterns of
an LLM, as these patterns are influenced by both the model architecture and the
pre-training dataset. Since the model and dataset are part of the solution
domain, not the problem domain where the input belongs, the outcome may differ
when the model is pre-trained on a different dataset. We propose SlimCode, a
model-agnostic code simplification solution for LLMs that depends on the nature
of input code tokens. As an empirical study on the LLMs including CodeBERT,
CodeT5, and GPT-4 for two main tasks: code search and summarization, we
reported that 1) the removal ratio of code has a linear-like relation with the
saving ratio on training time, 2) the impact of categorized tokens on code
simplification can vary significantly, 3) the impact of categorized tokens on
code simplification is task-specific but model-agnostic, and 4) the above
findings hold for the paradigm-prompt engineering and interactive in-context
learning. The empirical results showed that SlimCode can improve the
state-of-the-art technique by 9.46% and 5.15% in terms of MRR and BLEU score on
code search and summarization. Moreover, SlimCode is 133 times faster than the
state-of-the-art approach. Additionally, SlimCode can reduce the cost of
invoking GPT-4 by up to 24% per API query, while still producing comparable
results to those with the original code.
