  This paper introduces a dataset for training and evaluating methods for 6D
pose estimation of hand-held tools in task demonstrations captured by a
standard RGB camera. Despite the significant progress of 6D pose estimation
methods, their performance is usually limited for heavily occluded objects,
which is a common case in imitation learning, where the object is typically
partially occluded by the manipulating hand. Currently, there is a lack of
datasets that would enable the development of robust 6D pose estimation methods
for these conditions. To overcome this problem, we collect a new dataset
(Imitrob) aimed at 6D pose estimation in imitation learning and other
applications where a human holds a tool and performs a task. The dataset
contains image sequences of nine different tools and twelve manipulation tasks
with two camera viewpoints, four human subjects, and left/right hand. Each
image is accompanied by an accurate ground truth measurement of the 6D object
pose obtained by the HTC Vive motion tracking device. The use of the dataset is
demonstrated by training and evaluating a recent 6D object pose estimation
method (DOPE) in various setups.
