  Reliable autonomous navigation requires adapting the control policy of a
mobile robot in response to dynamics changes in different operational
conditions. Hand-designed dynamics models may struggle to capture model
variations due to a limited set of parameters. Data-driven dynamics learning
approaches offer higher model capacity and better generalization but require
large amounts of state-labeled data. This paper develops an approach for
learning robot dynamics directly from point-cloud observations, removing the
need and associated errors of state estimation, while embedding Hamiltonian
structure in the dynamics model to improve data efficiency. We design an
observation-space loss that relates motion prediction from the dynamics model
with motion prediction from point-cloud registration to train a Hamiltonian
neural ordinary differential equation. The learned Hamiltonian model enables
the design of an energy-shaping model-based tracking controller for rigid-body
robots. We demonstrate dynamics learning and tracking control on a real
nonholonomic wheeled robot.
