  Realtime perceptual and interaction capabilities in mixed reality require a
range of 3D tracking problems to be solved at low latency on
resource-constrained hardware such as head-mounted devices. Indeed, for devices
such as HoloLens 2 where the CPU and GPU are left available for applications,
multiple tracking subsystems are required to run on a continuous, real-time
basis while sharing a single Digital Signal Processor. To solve model-fitting
problems for HoloLens 2 hand tracking, where the computational budget is
approximately 100 times smaller than an iPhone 7, we introduce a new surface
model: the `Phong surface'. Using ideas from computer graphics, the Phong
surface describes the same 3D shape as a triangulated mesh model, but with
continuous surface normals which enable the use of lifting-based optimization,
providing significant efficiency gains over ICP-based methods. We show that
Phong surfaces retain the convergence benefits of smoother surface models,
while triangle meshes do not.
