  Stochastic learning dynamics based on Langevin or Levy stochastic
differential equations (SDEs) in deep neural networks control the variance of
noise by varying the size of the mini-batch or directly those of injecting
noise. Since the noise variance affects the approximation performance, the
design of the additive noise is significant in SDE-based learning and practical
implementation. In this paper, we propose an alternative stochastic descent
learning equation based on quantized optimization for non-convex objective
functions, adopting a stochastic analysis perspective. The proposed method
employs a quantized optimization approach that utilizes Langevin SDE dynamics,
allowing for controllable noise with an identical distribution without the need
for additive noise or adjusting the mini-batch size. Numerical experiments
demonstrate the effectiveness of the proposed algorithm on vanilla convolution
neural network(CNN) models and the ResNet-50 architecture across various data
sets. Furthermore, we provide a simple PyTorch implementation of the proposed
algorithm.
