  Security testing verifies that the data and the resources of software systems
are protected from attackers. Unfortunately, it suffers from the oracle
problem, which refers to the challenge, given an input for a system, of
distinguishing correct from incorrect behavior. In many situations where
potential vulnerabilities are tested, a test oracle may not exist, or it might
be impractical due to the many inputs for which specific oracles have to be
defined. In this paper, we propose a metamorphic testing approach that
alleviates the oracle problem in security testing. It enables engineers to
specify metamorphic relations (MRs) that capture security properties of the
system. Such MRs are then used to automate testing and detect vulnerabilities.
We provide a catalog of 22 system-agnostic MRs to automate security testing in
Web systems. Our approach targets 39% of the OWASP security testing activities
not automated by state-of-the-art techniques. It automatically detected 10 out
of 12 vulnerabilities affecting two widely used systems, one commercial and the
other open source (Jenkins).
