  Rigorous mathematical investigation of learning rates used in
back-propagation in shallow neural networks has become a necessity. This is
because experimental evidence needs to be endorsed by a theoretical background.
Such theory may be helpful in reducing the volume of experimental effort to
accomplish desired results. We leveraged the functional property of Mean Square
Error, which is Lipschitz continuous to compute learning rate in shallow neural
networks. We claim that our approach reduces tuning efforts, especially when a
significant corpus of data has to be handled. We achieve remarkable improvement
in saving computational cost while surpassing prediction accuracy reported in
literature. The learning rate, proposed here, is the inverse of the Lipschitz
constant. The work results in a novel method for carrying out gene expression
inference on large microarray data sets with a shallow architecture constrained
by limited computing resources. A combination of random sub-sampling of the
dataset, an adaptive Lipschitz constant inspired learning rate and a new
activation function, A-ReLU helped accomplish the results reported in the
paper.
