  Background: Developers spend a lot of their time on understanding source
code. Static code analysis tools can draw attention to code that is difficult
for developers to understand. However, most of the findings are based on
non-validated metrics, which can lead to confusion and code, that is hard to
understand, not being identified.
  Aims: In this work, we validate a metric called Cognitive Complexity which
was explicitly designed to measure code understandability and which is already
widely used due to its integration in well-known static code analysis tools.
  Method: We conducted a systematic literature search to obtain data sets from
studies which measured code understandability. This way we obtained about
24,000 understandability evaluations of 427 code snippets. We calculated the
correlations of these measurements with the corresponding metric values and
statistically summarized the correlation coefficients through a meta-analysis.
  Results: Cognitive Complexity positively correlates with comprehension time
and subjective ratings of understandability. The metric showed mixed results
for the correlation with the correctness of comprehension tasks and with
physiological measures.
  Conclusions: It is the first validated and solely code-based metric which is
able to reflect at least some aspects of code understandability. Moreover, due
to its methodology, this work shows that code understanding is currently
measured in many different ways, which we also do not know how they are
related. This makes it difficult to compare the results of individual studies
as well as to develop a metric that measures code understanding in all its
facets.
