  Crowd counting is an important yet challenging task in computer vision due to
serious occlusions, complex background and large scale variations, etc.
Multi-column architecture is widely adopted to overcome these challenges,
yielding state-of-the-art performance in many public benchmarks. However, there
still are two issues in such design: scale limitation and feature similarity.
Further performance improvements are thus restricted. In this paper, we propose
a novel crowd counting framework called Pyramid Scale Network (PSNet) to
explicitly address these issues. Specifically, for scale limitation, we adopt
three Pyramid Scale Modules (PSM) to efficiently capture multi-scale features,
which integrate a message passing mechanism and an attention mechanism into
multi-column architecture. Moreover, for feature similarity, a novel loss
function named Multi-column variance loss is introduced to make the features
learned by each column in PSM appropriately different from each other. To the
best of our knowledge, PSNet is the first work to explicitly address scale
limitation and feature similarity in multi-column design. Extensive experiments
on five benchmark datasets demonstrate the effectiveness of the proposed
innovations as well as the superior performance over the state-of-the-art. Our
code is publicly available at: https://github.com/oahunc/Pyramid_Scale_Network
