  Convolutional Neural Networks (CNNs) specialize in feature extraction rather
than function mapping. In doing so they form complex internal hierarchical
feature representations, the complexity of which gradually increases with a
corresponding increment in neural network depth. In this paper, we examine the
feature extraction capabilities of CNNs using Maximum Entropy (ME) and
Signal-to-Noise Ratio (SNR) to validate the idea that, CNN models should be
tailored for a given task and complexity of the input data. SNR and ME measures
are used as they can accurately determine in the input dataset, the relative
amount of signal information to the random noise and the maximum amount of
information respectively. We use two well known benchmarking datasets, MNIST
and CIFAR-10 to examine the information extraction and abstraction capabilities
of CNNs. Through our experiments, we examine convolutional feature extraction
and abstraction capabilities in CNNs and show that the classification accuracy
or performance of CNNs is greatly dependent on the amount, complexity and
quality of the signal information present in the input data. Furthermore, we
show the effect of information overflow and underflow on CNN classification
accuracies. Our hypothesis is that the feature extraction and abstraction
capabilities of convolutional layers are limited and therefore, CNN models
should be tailored to the input data by using appropriately sized CNNs based on
the SNR and ME measures of the input dataset.
