  We present a method for recognition of isolated Swedish Sign Language signs.
The method will be used in a game intended to help children training signing at
home, as a complement to training with a teacher. The target group is not
primarily deaf children, but children with language disorders. Using sign
language as a support in conversation has been shown to greatly stimulate the
speech development of such children. The signer is captured with an RGB-D
(Kinect) sensor, which has three advantages over a regular RGB camera. Firstly,
it allows complex backgrounds to be removed easily. We segment the hands and
face based on skin color and depth information. Secondly, it helps with the
resolution of hand over face occlusion. Thirdly, signs take place in 3D; some
aspects of the signs are defined by hand motion vertically to the image plane.
This motion can be estimated if the depth is observable. The 3D motion of the
hands relative to the torso are used as a cue together with the hand shape, and
HMMs trained with this input are used for classification. To obtain higher
robustness towards differences across signers, Fisher Linear Discriminant
Analysis is used to find the combinations of features that are most descriptive
for each sign, regardless of signer. Experiments show that the system can
distinguish signs from a challenging 94 word vocabulary with a precision of up
to 94% in the signer dependent case and up to 47% in the signer independent
case.
