  In this paper, we establish a theoretical connection between the classical
Lucas & Kanade (LK) algorithm and the emerging topic of Spatial Transformer
Networks (STNs). STNs are of interest to the vision and learning communities
due to their natural ability to combine alignment and classification within the
same theoretical framework. Inspired by the Inverse Compositional (IC) variant
of the LK algorithm, we present Inverse Compositional Spatial Transformer
Networks (IC-STNs). We demonstrate that IC-STNs can achieve better performance
than conventional STNs with less model capacity; in particular, we show
superior performance in pure image alignment tasks as well as joint
alignment/classification problems on real-world problems.
