  Since local feature detection has been one of the most active research areas
in computer vision during the last decade, a large number of detectors have
been proposed. The interest in feature-based applications continues to grow and
has thus rendered the task of characterizing the performance of various feature
detection methods an important issue in vision research. Inspired by the good
practices of electronic system design, a generic framework based on the
repeatability measure is presented in this paper that allows assessment of the
upper and lower bounds of detector performance and finds statistically
significant performance differences between detectors as a function of image
transformation amount by introducing a new variant of McNemars test in an
effort to design more reliable and effective vision systems. The proposed
framework is then employed to establish operating and guarantee regions for
several state-of-the-art detectors and to identify their statistical
performance differences for three specific image transformations: JPEG
compression, uniform light changes and blurring. The results are obtained using
a newly acquired, large image database (20482) images with 539 different
scenes. These results provide new insights into the behaviour of detectors and
are also useful from the vision systems design perspective.
