  Deep neural networks (DNNs) have made great progress in various fields. In
particular, the quantized neural network is a promising technique making DNNs
compatible on resource-limited devices for memory and computation saving. In
this paper, we mainly consider a non-convex minimization model with three
blocks to train quantized DNNs and propose a new stochastic three-block
alternating minimization (STAM) algorithm to solve it. We develop a convergence
theory for the STAM algorithm and obtain an $\epsilon$-stationary point with
optimal convergence rate $\mathcal{O}(\epsilon^{-4})$. Furthermore, we apply
our STAM algorithm to train DNNs with relaxed binary weights. The experiments
are carried out on three different network structures, namely VGG-11, VGG-16
and ResNet-18. These DNNs are trained using two different data sets, CIFAR-10
and CIFAR-100, respectively. We compare our STAM algorithm with some classical
efficient algorithms for training quantized neural networks. The test accuracy
indicates the effectiveness of STAM algorithm for training relaxed binary
quantization DNNs.
