  It is hard to densely track a nonrigid object in long term, which is a
fundamental research issue in the computer vision community. This task often
relies on estimating pairwise correspondences between images over time where
the error is accumulated and leads to a drift issue. In this paper, we
introduce a novel optimization framework with an Anchor Patch constraint. It is
supposed to significantly reduce overall errors given long sequences containing
non-rigidly deformable objects. Our framework can be applied to any dense
tracking algorithm, e.g. optical flow. We demonstrate the success of our
approach by showing significant error reduction on 6 popular optical flow
algorithms applied to a range of real-world nonrigid benchmarks. We also
provide quantitative analysis of our approach given synthetic occlusions and
image noise.
