  We study the problem of recognition of fingerspelled letter sequences in
American Sign Language in a signer-independent setting. Fingerspelled sequences
are both challenging and important to recognize, as they are used for many
content words such as proper nouns and technical terms. Previous work has shown
that it is possible to achieve almost 90% accuracies on fingerspelling
recognition in a signer-dependent setting. However, the more realistic
signer-independent setting presents challenges due to significant variations
among signers, coupled with the dearth of available training data. We
investigate this problem with approaches inspired by automatic speech
recognition. We start with the best-performing approaches from prior work,
based on tandem models and segmental conditional random fields (SCRFs), with
features based on deep neural network (DNN) classifiers of letters and
phonological features. Using DNN adaptation, we find that it is possible to
bridge a large part of the gap between signer-dependent and signer-independent
performance. Using only about 115 transcribed words for adaptation from the
target signer, we obtain letter accuracies of up to 82.7% with frame-level
adaptation labels and 69.7% with only word labels.
