  A dataset is crucial for model learning and evaluation. Choosing the right
dataset to use or making a new dataset requires the knowledge of those that are
available. In this work, we provide that knowledge, by reviewing twenty
datasets that were published in the recent six years and that are directly
related to object manipulation. We report on modalities, activities, and
annotations for each individual dataset and give our view on its use for object
manipulation. We also compare the datasets and summarize them. We conclude with
our suggestion on future datasets.
