  Apprenticeship learning has recently attracted a wide attention due to its
capability of allowing robots to learn physical tasks directly from
demonstrations provided by human experts. Most previous techniques assumed that
the state space is known a priori or employed simple state representations that
usually suffer from perceptual aliasing. Different from previous research, we
propose a novel approach named Sequence-based Multimodal Apprenticeship
Learning (SMAL), which is capable to simultaneously fusing temporal information
and multimodal data, and to integrate robot perception with decision making. To
evaluate the SMAL approach, experiments are performed using both simulations
and real-world robots in the challenging search and rescue scenarios. The
empirical study has validated that our SMAL approach can effectively learn
plans for robots to make decisions using sequence of multimodal observations.
Experimental results have also showed that SMAL outperforms the baseline
methods using individual images.
