  American Sign Language recognition is a difficult gesture recognition
problem, characterized by fast, highly articulate gestures. These are comprised
of arm movements with different hand shapes, facial expression and head
movements. Among these components, hand shape is the vital, often the most
discriminative part of a gesture. In this work, we present an approach for
effective learning of hand shape embeddings, which are discriminative for ASL
gestures. For hand shape recognition our method uses a mix of manually labelled
hand shapes and high confidence predictions to train deep convolutional neural
network (CNN). The sequential gesture component is captured by recursive neural
network (RNN) trained on the embeddings learned in the first stage. We will
demonstrate that higher quality hand shape models can significantly improve the
accuracy of final video gesture classification in challenging conditions with
variety of speakers, different illumination and significant motion blurr. We
compare our model to alternative approaches exploiting different modalities and
representations of the data and show improved video gesture recognition
accuracy on GMU-ASL51 benchmark dataset
