  Citation based country rankings consistently categorize Japan as a developing
country, even in those from the most reputed institutions. This categorization
challenges the credibility of such rankings, considering Japan elevated
scientific standing. In most cases, these rankings use percentile indicators
and are accurate if country citations fit an ideal model of distribution, but
they can be misleading in cases of deviations. The ideal model implies a
lognormal citation distribution and a power law citation based double rank: in
the global and country lists. This report conducts a systematic examination of
deviations from the ideal model and their consequential impact on evaluations.
The study evaluates six selected countries across three scientifically relevant
topics and utilizes Leiden Ranking assessments of over 300 universities. The
findings reveal three types of deviations from the lognormal citation
distribution: i deviations in the extreme upper tail; ii inflated lower tails;
and iii deflated lower part of the distributions. These deviations stem from
structural differences among research systems that are prevalent and have the
potential to mislead evaluations across all research levels. Consequently,
reliable evaluations must consider these deviations. Otherwise, while some
countries and institutions will be correctly evaluated, failure to identify
deviations in each specific country or institution will render uncertain
evaluations. For reliable assessments, future research evaluations of countries
and institutions must identify deviations from the ideal model.
