  Purpose: Tracking the 3D motion of the surgical tool and the patient anatomy
is a fundamental requirement for computer-assisted skull-base surgery. The
estimated motion can be used both for intra-operative guidance and for
downstream skill analysis. Recovering such motion solely from surgical videos
is desirable, as it is compliant with current clinical workflows and
instrumentation.
  Methods: We present Tracker of Anatomy and Tool (TAToo). TAToo jointly tracks
the rigid 3D motion of patient skull and surgical drill from stereo microscopic
videos. TAToo estimates motion via an iterative optimization process in an
end-to-end differentiable form. For robust tracking performance, TAToo adopts a
probabilistic formulation and enforces geometric constraints on the object
level.
  Results: We validate TAToo on both simulation data, where ground truth motion
is available, as well as on anthropomorphic phantom data, where optical
tracking provides a strong baseline. We report sub-millimeter and millimeter
inter-frame tracking accuracy for skull and drill, respectively, with rotation
errors below 1{\deg}. We further illustrate how TAToo may be used in a surgical
navigation setting.
  Conclusion: We present TAToo, which simultaneously tracks the surgical tool
and the patient anatomy in skull-base surgery. TAToo directly predicts the
motion from surgical videos, without the need of any markers. Our results show
that the performance of TAToo compares favorably to competing approaches.
Future work will include fine-tuning of our depth network to reach a 1 mm
clinical accuracy goal desired for surgical applications in the skull base.
