  In the framework of convolutional neural networks that lie at the heart of
deep learning, downsampling is often performed with a max-pooling operation
that only retains the element with maximum activation, while completely
discarding the information contained in other elements in a pooling region. To
address this issue, a novel pooling scheme, Ordinal Pooling Network (OPN), is
introduced in this work. OPN rearranges all the elements of a pooling region in
a sequence and assigns different weights to these elements based upon their
orders in the sequence, where the weights are learned via the gradient-based
optimisation. The results of our small-scale experiments on image
classification task demonstrate that this scheme leads to a consistent
improvement in the accuracy over max-pooling operation. This improvement is
expected to increase in deeper networks, where several layers of pooling become
necessary.
