  Mutation testing consists of evaluating how effective test suites are at
detecting artificially seeded defects in the source code, and guiding the
improvement of the test suites. Although mutation testing tools are
increasingly adopted in practice, equivalent mutants, i.e., mutants that differ
only in syntax but not semantics, hamper this process. While prior research
investigated how frequently equivalent mutants are produced by mutation testing
tools and how effective existing methods of detecting these equivalent mutants
are, it remains unclear to what degree humans also create equivalent mutants,
and how well they perform at identifying these. We therefore study these
questions in the context of Code Defenders, a mutation testing game, in which
players competitively produce mutants and tests. Using manual inspection as
well as automated identification methods we establish that less than 10 % of
manually created mutants are equivalent. Surprisingly, our findings indicate
that a significant portion of developers struggle to accurately identify
equivalent mutants, emphasizing the need for improved detection mechanisms and
developer training in mutation testing.
