  A recent line of work has shown that end-to-end optimization of Bayesian
filters can be used to learn state estimators for systems whose underlying
models are difficult to hand-design or tune, while retaining the core
advantages of probabilistic state estimation. As an alternative approach for
state estimation in these settings, we present an end-to-end approach for
learning state estimators modeled as factor graph-based smoothers. By unrolling
the optimizer we use for maximum a posteriori inference in these probabilistic
graphical models, we can learn probabilistic system models in the full context
of an overall state estimator, while also taking advantage of the distinct
accuracy and runtime advantages that smoothers offer over recursive filters. We
study this approach using two fundamental state estimation problems, object
tracking and visual odometry, where we demonstrate a significant improvement
over existing baselines. Our work comes with an extensive code release, which
includes training and evaluation scripts, as well as Python libraries for Lie
theory and factor graph optimization:
https://sites.google.com/view/diffsmoothing/
