  Testing is an important aspect of software development, but unfortunately, it
is often neglected. While test quality analyses such as code coverage or
mutation analysis inform developers about the quality of their tests, such
reports are viewed only sporadically during continuous integration or code
review, if they are considered at all, and their impact on the developers'
testing behavior therefore tends to be negligible. To actually influence
developer behavior, it may rather be necessary to motivate developers directly
within their programming environment, while they are coding. We introduce
IntelliGame, a gamified plugin for the popular IntelliJ Java Integrated
Development Environment, which rewards developers for positive testing behavior
using a multi-level achievement system: A total of 27 different achievements,
each with incremental levels, provide affirming feedback when developers
exhibit commendable testing behavior, and provide an incentive to further
continue and improve this behavior. A controlled experiment with 49
participants given a Java programming task reveals substantial differences in
the testing behavior triggered by IntelliGame: Incentivized developers write
more tests, achieve higher coverage and mutation scores, run their tests more
often, and achieve functionality earlier.
