  We propose a novel multi-scale template matching method which is robust
against both scaling and rotation in unconstrained environments. The key
component behind is a similarity measure referred to as scalable diversity
similarity (SDS). Specifically, SDS exploits bidirectional diversity of the
nearest neighbor (NN) matches between two sets of points. To address the
scale-robustness of the similarity measure, local appearance and rank
information are jointly used for the NN search. Furthermore, by introducing
penalty term on the scale change, and polar radius term into the similarity
measure, SDS is shown to be a well-performing similarity measure against
overall size and rotation changes, as well as non-rigid geometric deformations,
background clutter, and occlusions. The properties of SDS are statistically
justified, and experiments on both synthetic and real-world data show that SDS
can significantly outperform state-of-the-art methods.
