  Consistency, defined as the requirement that a series of measurements of the
same project carried out by different raters using the same method should
produce similar results, is one of the most important aspects to be taken into
account in the measurement methods of the software. In spite of this, there is
a widespread view that many measurement methods introduce an undesirable amount
of subjectivity in the measurement process. This perception has made several
organizations develop revisions of the standard methods whose main aim is to
improve their consistency by introducing some suitable modifications of those
aspects which are believed to introduce a greater degree of subjectivity.Each
revision of a method must be empirically evaluated to determine to what extent
is the aim of improving its consistency achieved. In this article we will
define an homogeneous statistic intended to describe the consistency level of a
method, and we will develop the statistical analysis which should be carried
out in order to conclude whether or not a measurement method is more consistent
than other one.
