  This research embarks on pioneering the integration of gradient sampling
optimization techniques, particularly StochGradAdam, into the pruning process
of neural networks. Our main objective is to address the significant challenge
of maintaining accuracy in pruned neural models, critical in
resource-constrained scenarios. Through extensive experimentation, we
demonstrate that gradient sampling significantly preserves accuracy during and
after the pruning process compared to traditional optimization methods. Our
study highlights the pivotal role of gradient sampling in robust learning and
maintaining crucial information post substantial model simplification. The
results across CIFAR-10 datasets and residual neural architectures validate the
versatility and effectiveness of our approach. This work presents a promising
direction for developing efficient neural networks without compromising
performance, even in environments with limited computational resources.
