  Understanding hand-object pose with computer vision opens the door to new
applications in mixed reality, assisted living or human-robot interaction. Most
methods are trained and evaluated on balanced datasets. This is of limited use
in real-world applications; how do these methods perform in the wild on unknown
objects? We propose a novel benchmark for object group distribution shifts in
hand and object pose regression. We then test the hypothesis that meta-learning
a baseline pose regression neural network can adapt to these shifts and
generalize better to unknown objects. Our results show measurable improvements
over the baseline, depending on the amount of prior knowledge. For the task of
joint hand-object pose regression, we observe optimization interference for the
meta-learner. To address this issue and improve the method further, we provide
a comprehensive analysis which should serve as a basis for future work on this
benchmark.
