  Human face images usually appear with wide range of visual scales. The
existing face representations pursue the bandwidth of handling scale variation
via multi-scale scheme that assembles a finite series of predefined scales.
Such multi-shot scheme brings inference burden, and the predefined scales
inevitably have gap from real data. Instead, learning scale parameters from
data, and using them for one-shot feature inference, is a decent solution. To
this end, we reform the conv layer by resorting to the scale-space theory, and
achieve two-fold facilities: 1) the conv layer learns a set of scales from real
data distribution, each of which is fulfilled by a conv kernel; 2) the layer
automatically highlights the feature at the proper channel and location
corresponding to the input pattern scale and its presence. Then, we accomplish
the hierarchical scale attention by stacking the reformed layers, building a
novel style named SCale AttentioN Conv Neural Network (\textbf{SCAN-CNN}). We
apply SCAN-CNN to the face recognition task and push the frontier of SOTA
performance. The accuracy gain is more evident when the face images are blurry.
Meanwhile, as a single-shot scheme, the inference is more efficient than
multi-shot fusion. A set of tools are made to ensure the fast training of
SCAN-CNN and zero increase of inference cost compared with the plain CNN.
