  In this paper, we consider the problem of learning object manipulation tasks
from human demonstration using RGB or RGB-D cameras. We highlight the key
challenges in capturing sufficiently good data with no tracking devices -
starting from sensor selection and accurate 6DoF pose estimation to natural
language processing. In particular, we focus on two showcases: gluing task with
a glue gun and simple block-stacking with variable blocks. Furthermore, we
discuss how a linguistic description of the task could help to improve the
accuracy of task description. We also present the whole architecture of our
transfer of the imitated task to the simulated and real robot environment.
