  Recently, deep learning becomes the main focus of machine learning research
and has greatly impacted many important fields. However, deep learning is
criticized for lack of interpretability. As a successful unsupervised model in
deep learning, the autoencoder embraces a wide spectrum of applications, yet it
suffers from the model opaqueness as well. In this paper, we propose a new type
of convolutional autoencoders, termed as Soft Autoencoder (Soft-AE), in which
the activation functions of encoding layers are implemented with adaptable
soft-thresholding units while decoding layers are realized with linear units.
Consequently, Soft-AE can be naturally interpreted as a learned cascaded
wavelet shrinkage system. Our denoising experiments demonstrate that Soft-AE
not only is interpretable but also offers a competitive performance relative to
its counterparts. Furthermore, we propose a generalized linear unit (GenLU) to
make an autoencoder more adaptive in nonlinearly filtering images and data,
such as denoising and deblurring.
