  Fingerspelling in sign language has been the means of communicating technical
terms and proper nouns when they do not have dedicated sign language gestures.
Automatic recognition of fingerspelling can help resolve communication barriers
when interacting with deaf people. The main challenges prevalent in
fingerspelling recognition are the ambiguity in the gestures and strong
articulation of the hands. The automatic recognition model should address high
inter-class visual similarity and high intra-class variation in the gestures.
Most of the existing research in fingerspelling recognition has focused on the
dataset collected in a controlled environment. The recent collection of a
large-scale annotated fingerspelling dataset in the wild, from social media and
online platforms, captures the challenges in a real-world scenario. In this
work, we propose a fine-grained visual attention mechanism using the
Transformer model for the sequence-to-sequence prediction task in the wild
dataset. The fine-grained attention is achieved by utilizing the change in
motion of the video frames (optical flow) in sequential context-based attention
along with a Transformer encoder model. The unsegmented continuous video
dataset is jointly trained by balancing the Connectionist Temporal
Classification (CTC) loss and the maximum-entropy loss. The proposed approach
can capture better fine-grained attention in a single iteration. Experiment
evaluations show that it outperforms the state-of-the-art approaches.
