  In parallel with the success of CNNs to solve vision problems, there is a
growing interest in developing methodologies to understand and visualize the
internal representations of these networks. How the responses of a trained CNN
encode the visual information is a fundamental question both for computer and
human vision research. Image representations provided by the first
convolutional layer as well as the resolution change provided by the
max-polling operation are easy to understand, however, as soon as a second and
further convolutional layers are added in the representation, any intuition is
lost. A usual way to deal with this problem has been to define deconvolutional
networks that somehow allow to explore the internal representations of the most
important activations towards the image space, where deconvolution is assumed
as a convolution with the transposed filter. However, this assumption is not
the best approximation of an inverse convolution. In this paper we propose a
new assumption based on filter substitution to reverse the encoding of a
convolutional layer. This provides us with a new tool to directly visualize any
CNN single neuron as a filter in the first layer, this is in terms of the image
space.
