  Soft-thresholding has been widely used in neural networks. Its basic network
structure is a two-layer convolution neural network with soft-thresholding. Due
to the network's nature of nonlinearity and nonconvexity, the training process
heavily depends on an appropriate initialization of network parameters,
resulting in the difficulty of obtaining a globally optimal solution. To
address this issue, a convex dual network is designed here. We theoretically
analyze the network convexity and numerically confirm that the strong duality
holds. This conclusion is further verified in the linear fitting and denoising
experiments. This work provides a new way to convexify soft-thresholding neural
networks.
