  Convolutional neural networks (CNNs) learn filters in order to capture local
correlation patterns in feature space. In contrast, in this paper we propose
harmonic blocks that produce features by learning optimal combinations of
spectral filters defined by the Discrete Cosine Transform. The harmonic blocks
are used to replace conventional convolutional layers to construct partial or
fully harmonic CNNs. We extensively validate our approach and show that the
introduction of harmonic blocks into state-of-the-art CNN baseline
architectures results in comparable or better performance in classification
tasks on small NORB, CIFAR10 and CIFAR100 datasets.
