  The entropy error function has been widely used in neural networks.
Nevertheless, the network training based on this error function generally leads
to a slow convergence rate, and can easily be trapped in a local minimum or
even with the incorrect saturation problem in practice. In fact, there are many
results based on entropy error function in neural network and its applications.
However, the theory of such an algorithm and its convergence have not been
fully studied so far. To tackle the issue, we propose a novel entropy function
with smoothing l0 regularization for feed-forward neural networks. Using
real-world datasets, we performed an empirical evaluation to demonstrate that
the newly conceived algorithm allows us to substantially improve the prediction
performance of the considered neural networks. More importantly, the
experimental results also show that our proposed function brings in more
precise classifications, compared to well-founded baselines. Our work is novel
as it enables neural networks to learn effectively, producing more accurate
predictions compared to state-of-the-art algorithms. In this respect, we expect
that the algorithm will contribute to existing studies in the field, advancing
research in Machine Learning and Deep Learning.
