  Neural network pruning has become increasingly crucial due to the complexity
of these models and their widespread use in various fields. Existing pruning
algorithms often suffer from limitations such as architecture specificity,
excessive complexity and reliance on demanding calculations, rendering them
impractical for real-world applications. This paper introduces KEN: a
straightforward, universal and unstructured pruning algorithm based on Kernel
Density Estimation (KDE). KEN aims to construct optimized transformers by
selectively preserving the most significant parameters while restoring others
to their pre-training state. This strategy preserves model performance while
enabling storage of only the optimized subnetwork, leading to substantial
memory savings. Extensive evaluations across seven different LLMs demonstrate
that KEN achieves equal or better performance than their original unpruned
versions, with a minimum parameter reduction of 25%. Furthermore, in-depth
comparisons with established pruning and PEFT algorithms confirm KEN
effectiveness. We further introduce KEN$_{viz}$, an explainable tool that
visualizes the optimized model composition achieved by KEN from different
points of view.
