  This paper presents a modular, extensible and highly efficient open source
framework for registration based tracking called Modular Tracking Framework
(MTF). Targeted at robotics applications, it is implemented entirely in C++ and
designed from the ground up to easily integrate with systems that support any
of several major vision and robotics libraries including OpenCV, ROS, ViSP and
Eigen. It implements more methods, is faster, and more precise than other
existing systems. Further, the theoretical basis for its design is a new way to
conceptualize registration based trackers that decomposes them into three
constituent sub modules - Search Method (SM), Appearance Model (AM) and State
Space Model (SSM).
  In the process, we integrate many important advances published after Baker \&
Matthews' landmark work in 2004. In addition to being a practical solution for
fast and high precision tracking, MTF can also serve as a useful research tool
by allowing existing and new methods for any of the sub modules to be studied
better. When a new method is introduced for one of these, the breakdown can
help to experimentally find the combination of methods for the others that is
optimum for it. By extensive use of generic programming, MTF makes it easy to
plug in a new method for any of the sub modules so that it can not only be
tested comprehensively with existing methods but also become immediately
available for deployment in any project that uses the framework. With 16 AMs,
11 SMs and 13 SSMs implemented already, MTF provides over 2000 distinct single
layer trackers. It also allows two or more of these to be combined together in
several ways to create a practically unlimited variety of novel multi layer
trackers.
