  Recently, dropout has seen increasing use in deep learning. For deep
convolutional neural networks, dropout is known to work well in fully-connected
layers. However, its effect in pooling layers is still not clear. This paper
demonstrates that max-pooling dropout is equivalent to randomly picking
activation based on a multinomial distribution at training time. In light of
this insight, we advocate employing our proposed probabilistic weighted
pooling, instead of commonly used max-pooling, to act as model averaging at
test time. Empirical evidence validates the superiority of probabilistic
weighted pooling. We also compare max-pooling dropout and stochastic pooling,
both of which introduce stochasticity based on multinomial distributions at
pooling stage.
