  All hand-object interaction is controlled by forces that the two bodies exert
on each other, but little work has been done in modeling these underlying
forces when doing pose and contact estimation from RGB/RGB-D data. Given the
pose of the hand and object from any pose estimation system, we propose an
end-to-end differentiable model that refines pose estimates by learning the
forces experienced by the object at each vertex in its mesh. By matching the
learned net force to an estimate of net force based on finite differences of
position, this model is able to find forces that accurately describe the
movement of the object, while resolving issues like mesh interpenetration and
lack of contact. Evaluating on the ContactPose dataset, we show this model
successfully corrects poses and finds contact maps that better match the ground
truth, despite not using any RGB or depth image data.
