  Triggered by the success of transformers in various visual tasks, the spatial
self-attention mechanism has recently attracted more and more attention in the
computer vision community. However, we empirically found that a typical vision
transformer with the spatial self-attention mechanism could not learn accurate
attention maps for distinguishing different categories of fine-grained images.
To address this problem, motivated by the temporal attention mechanism in
brains, we propose a spatial-temporal attention network for learning
fine-grained feature representations, called STAN, where the features learnt by
implementing a sequence of spatial self-attention operations corresponding to
multiple moments are aggregated progressively. The proposed STAN consists of
four modules: a self-attention backbone module for learning a sequence of
features with self-attention operations, a spatial feature self-organizing
module for facilitating the model training, a spatial-temporal feature learning
module for aggregating the re-organized features via a Long Short-Term Memory
network, and a context-aware module that is implemented as the forget block of
the spatial-temporal feature learning module for preserving/forgetting the
long-term memory by utilizing contextual information. Then, we propose a
STAN-based method for open-set fine-grained recognition by integrating the
proposed STAN network with a linear classifier, called STAN-OSFGR. Extensive
experimental results on 3 fine-grained datasets and 2 coarse-grained datasets
demonstrate that the proposed STAN-OSFGR outperforms 9 state-of-the-art
open-set recognition methods significantly in most cases.
