  National research evaluation exercises provide a comparative measure of
research performance of the nation's institutions, and as such represent a tool
for stimulating research productivity, particularly if the results are used to
inform selective funding by government. While a school of thought welcomes
frequent changes in evaluation criteria in order to prevent the subjects
evaluated from adopting opportunistic behaviors, it is evident that the "rules
of the game" should above all be functional towards policy objectives, and
therefore be known with adequate forewarning prior to the evaluation period.
Otherwise, the risk is that policy-makers will find themselves faced by a
dilemma: should they reward universities that responded best to the criteria in
effect at the outset of the observation period or those that result as best
according to rules that emerged during or after the observation period? This
study verifies if and to what extent some universities are penalized instead of
rewarded for good behavior, in pursuit of the objectives of the "known" rules
of the game, by comparing the research performances of Italian universities for
the period of the nation's next evaluation exercise (2004-2008): first as
measured according to criteria available at the outset of the period and next
according to those announced at the end of the period.
