  Due to the increasing sophistication of web attacks, Web Application
Firewalls (WAFs) have to be tested and updated regularly to resist the
relentless flow of web attacks. In practice, using a brute-force attack to
discover vulnerabilities is infeasible due to the wide variety of attack
patterns. Thus, various black-box testing techniques have been proposed in the
literature. However, these techniques suffer from low efficiency. This paper
presents Reinforcement-Learning-Driven and Adaptive Testing (RAT), an automated
black-box testing strategy to discover injection vulnerabilities in WAFs. In
particular, we focus on SQL injection and Cross-site Scripting, which have been
among the top ten vulnerabilities over the past decade. More specifically, RAT
clusters similar attack samples together. It then utilizes a reinforcement
learning technique combined with a novel adaptive search algorithm to discover
almost all bypassing attack patterns efficiently. We compare RAT with three
state-of-the-art methods considering their objectives. The experiments show
that RAT performs 33.53% and 63.16% on average better than its counterparts in
discovering the most possible bypassing payloads and reducing the number of
attempts before finding the first bypassing payload when testing
well-configured WAFs, respectively.
