  This paper presents a novel keypoints-based attention mechanism for visual
recognition in still images. Deep Convolutional Neural Networks (CNNs) for
recognizing images with distinctive classes have shown great success, but their
performance in discriminating fine-grained changes is not at the same level. We
address this by proposing an end-to-end CNN model, which learns meaningful
features linking fine-grained changes using our novel attention mechanism. It
captures the spatial structures in images by identifying semantic regions (SRs)
and their spatial distributions, and is proved to be the key to modelling
subtle changes in images. We automatically identify these SRs by grouping the
detected keypoints in a given image. The ``usefulness'' of these SRs for image
recognition is measured using our innovative attentional mechanism focusing on
parts of the image that are most relevant to a given task. This framework
applies to traditional and fine-grained image recognition tasks and does not
require manually annotated regions (e.g. bounding-box of body parts, objects,
etc.) for learning and prediction. Moreover, the proposed keypoints-driven
attention mechanism can be easily integrated into the existing CNN models. The
framework is evaluated on six diverse benchmark datasets. The model outperforms
the state-of-the-art approaches by a considerable margin using Distracted
Driver V1 (Acc: 3.39%), Distracted Driver V2 (Acc: 6.58%), Stanford-40 Actions
(mAP: 2.15%), People Playing Musical Instruments (mAP: 16.05%), Food-101 (Acc:
6.30%) and Caltech-256 (Acc: 2.59%) datasets.
