  We show how feature maps in convolutional networks are susceptible to spatial
bias. Due to a combination of architectural choices, the activation at certain
locations is systematically elevated or weakened. The major source of this bias
is the padding mechanism. Depending on several aspects of convolution
arithmetic, this mechanism can apply the padding unevenly, leading to
asymmetries in the learned weights. We demonstrate how such bias can be
detrimental to certain tasks such as small object detection: the activation is
suppressed if the stimulus lies in the impacted area, leading to blind spots
and misdetection. We propose solutions to mitigate spatial bias and demonstrate
how they can improve model accuracy.
