  Hand gestures can provide a natural means of human-computer interaction and
enable people who cannot speak to communicate efficiently. Existing hand
gesture recognition methods heavily depend on pre-defined gestures, however,
motor-impaired individuals require new gestures tailored to each individual's
gesture motion and style. Gesture samples collected from different persons have
distribution shifts due to their health conditions, the severity of the
disability, motion patterns of the arms, etc. In this paper, we introduce the
Latent Embedding Exploitation (LEE) mechanism in our replay-based Few-Shot
Continual Learning (FSCL) framework that significantly improves the performance
of fine-tuning a model for out-of-distribution data. Our method produces a
diversified latent feature space by leveraging a preserved latent embedding
known as gesture prior knowledge, along with intra-gesture divergence derived
from two additional embeddings. Thus, the model can capture latent statistical
structure in highly variable gestures with limited samples. We conduct an
experimental evaluation using the SmartWatch Gesture and the Motion Gesture
datasets. The proposed method results in an average test accuracy of 57.0%,
64.6%, and 69.3% by using one, three, and five samples for six different
gestures. Our method helps motor-impaired persons leverage wearable devices,
and their unique styles of movement can be learned and applied in
human-computer interaction and social communication. Code is available at:
https://github.com/riyadRafiq/wearable-latent-embedding-exploitation
