  Extending the abilities of service robots is important for expanding what
they can achieve in everyday manipulation tasks. On the other hand, it is also
essential to ensure them to determine what they can not achieve in certain
cases due to either anomalies or permanent failures during task execution.
Robots need to identify these situations, and reveal the reasons behind these
cases to overcome and recover from them. In this paper, we propose and analyze
a Long Short-Term Memories-based (LSTM-based) awareness approach to reveal the
reasons behind an anomaly case that occurs during a manipulation episode in an
unstructured environment. The proposed method takes into account the real-time
observations of the robot by fusing visual, auditory and proprioceptive sensory
modalities to achieve this task. We also provide a comparative analysis of our
method with Hidden Markov Models (HMMs) and Conditional Random Fields (CRFs).
The symptoms of anomalies are first learned from a given training set, then
they can be classified in real-time based on the learned models. The approaches
are evaluated on a Baxter robot executing object manipulation scenarios. The
results indicate that the LSTM-based method outperforms the other methods with
a 0.94 classification rate in revealing causes of anomalies in case of an
unexpected deviation.
