  Explainable artificial intelligence (AI) techniques are increasingly being
explored to provide insights into why AI and machine learning (ML) models
provide a certain outcome in various applications. However, there has been
limited exploration of explainable AI techniques on time-series data,
especially in the healthcare context. In this paper, we describe a
threshold-based method that utilizes a weakly supervised model and a
gradient-based explainable AI technique (i.e. saliency map) and explore its
feasibility to identify salient frames of time-series data. Using the dataset
from 15 post-stroke survivors performing three upper-limb exercises and labels
on whether a compensatory motion is observed or not, we implemented a
feed-forward neural network model and utilized gradients of each input on model
outcomes to identify salient frames that involve compensatory motions.
According to the evaluation using frame-level annotations, our approach
achieved a recall of 0.96 and an F2-score of 0.91. Our results demonstrated the
potential of a gradient-based explainable AI technique (e.g. saliency map) for
time-series data, such as highlighting the frames of a video that therapists
should focus on reviewing and reducing the efforts on frame-level labeling for
model training.
