  Large language models(LLMs) containing tens of billions of parameters (or
even more) have demonstrated impressive capabilities in various NLP tasks.
However, substantial model size poses challenges to training, inference, and
deployment so that it is necessary to compress the model. At present, most
model compression for LLMs requires manual design of pruning features, which
has problems such as complex optimization pipeline and difficulty in retaining
the capabilities of certain parts of the model.Therefore, we propose a novel
pruning approach: firstly, a training set of a certain number of
architecture-accuracy pairs is established, and then a non-neural model is
trained as an accuracy predictor. Using the accuracy predictor to further
optimize the search space and search, the optimal model can be automatically
selected. Experiments show that our proposed approach is effective and
efficient. Compared with the baseline, the perplexity(PPL) on Wikitext2 and PTB
dropped by 9.48% and 5,76% respectively, and the average accuracy of MMLU
increased by 6.28%.
