  In this paper, we propose a generalized framework for developing
learning-rate-free momentum stochastic gradient descent (SGD) methods in the
minimization of nonsmooth nonconvex functions, especially in training nonsmooth
neural networks. Our framework adaptively generates learning rates based on the
historical data of stochastic subgradients and iterates. Under mild conditions,
we prove that our proposed framework enjoys global convergence to the
stationary points of the objective function in the sense of the conservative
field, hence providing convergence guarantees for training nonsmooth neural
networks. Based on our proposed framework, we propose a novel
learning-rate-free momentum SGD method (LFM). Preliminary numerical experiments
reveal that LFM performs comparably to the state-of-the-art learning-rate-free
methods (which have not been shown theoretically to be convergence) across
well-known neural network training benchmarks.
