  Keypoint annotation in point clouds is an important task for 3D
reconstruction, object tracking and alignment, in particular in deformable or
moving scenes. In the context of agriculture robotics, it is a critical task
for livestock automation to work toward condition assessment or behaviour
recognition. In this work, we propose a novel approach for semantic keypoint
annotation in point clouds, by reformulating the keypoint extraction as a
regression problem of the distance between the keypoints and the rest of the
point cloud. We use the distance on the point cloud manifold mapped into a
radial basis function (RBF), which is then learned using an encoder-decoder
architecture. Special consideration is given to the data augmentation specific
to multi-depth-camera systems by considering noise over the extrinsic
calibration and camera frame dropout. Additionally, we investigate
computationally efficient non-rigid deformation methods that can be applied to
animal point clouds. Our method is tested on data collected in the field, on
moving beef cattle, with a calibrated system of multiple hardware-synchronised
RGB-D cameras.
