  In this paper, we propose a novel Discrete Cosine Transform (DCT)-based
neural network layer which we call DCT-perceptron to replace the $3\times3$
Conv2D layers in the Residual neural Network (ResNet). Convolutional filtering
operations are performed in the DCT domain using element-wise multiplications
by taking advantage of the Fourier and DCT Convolution theorems. A trainable
soft-thresholding layer is used as the nonlinearity in the DCT perceptron.
Compared to ResNet's Conv2D layer which is spatial-agnostic and
channel-specific, the proposed layer is location-specific and channel-specific.
The DCT-perceptron layer reduces the number of parameters and multiplications
significantly while maintaining comparable accuracy results of regular ResNets
in CIFAR-10 and ImageNet-1K. Moreover, the DCT-perceptron layer can be inserted
with a batch normalization layer before the global average pooling layer in the
conventional ResNets as an additional layer to improve classification accuracy.
