  Background: Construct validity concerns the use of indicators to measure a
concept that is not directly measurable. Aim: This study intends to identify,
categorize, assess and quantify discussions of threats to construct validity in
empirical software engineering literature and use the findings to suggest ways
to improve the reporting of construct validity issues. Method: We analyzed 83
articles that report human-centric experiments published in five top-tier
software engineering journals from 2015 to 2019. The articles' text concerning
threats to construct validity was divided into segments (the unit of analysis)
based on predefined categories. The segments were then evaluated regarding
whether they clearly discussed a threat and a construct. Results: Three-fifths
of the segments were associated with topics not related to construct validity.
Two-thirds of the articles discussed construct validity without using the
definition of construct validity given in the article. The threats were clearly
described in more than four-fifths of the segments, but the construct in
question was clearly described in only two-thirds of the segments. The
construct was unclear when the discussion was not related to construct validity
but to other types of validity. Conclusions: The results show potential for
improving the understanding of construct validity in software engineering.
Recommendations addressing the identified weaknesses are given to improve the
awareness and reporting of CV.
