  Sign Language Recognition (SLR) systems aim to be embedded in video stream
platforms to recognize the sign performed in front of a camera. SLR research
has taken advantage of recent advances in pose estimation models to use
skeleton sequences estimated from videos instead of RGB information to predict
signs. This approach can make HAR-related tasks less complex and more robust to
diverse backgrounds, lightning conditions, and physical appearances. In this
work, we explore the use of a spatio-temporal skeleton representation such as
Tree Structure Skeleton Image (TSSI) as an alternative input to improve the
accuracy of skeleton-based models for SLR. TSSI converts a skeleton sequence
into an RGB image where the columns represent the joints of the skeleton in a
depth-first tree traversal order, the rows represent the temporal evolution of
the joints, and the three channels represent the (x, y, z) coordinates of the
joints. We trained a DenseNet-121 using this type of input and compared it with
other skeleton-based deep learning methods using a large-scale American Sign
Language (ASL) dataset, WLASL. Our model (SL-TSSI-DenseNet) overcomes the
state-of-the-art of other skeleton-based models. Moreover, when including data
augmentation our proposal achieves better results than both skeleton-based and
RGB-based models. We evaluated the effectiveness of our model on the Ankara
University Turkish Sign Language (TSL) dataset, AUTSL, and a Mexican Sign
Language (LSM) dataset. On the AUTSL dataset, the model achieves similar
results to the state-of-the-art of other skeleton-based models. On the LSM
dataset, the model achieves higher results than the baseline. Code has been
made available at: https://github.com/davidlainesv/SL-TSSI-DenseNet.
