  Website hacking is a frequent attack type used by malicious actors to obtain
confidential information, modify the integrity of web pages or make websites
unavailable. The tools used by attackers are becoming more and more automated
and sophisticated, and malicious machine learning agents seems to be the next
development in this line. In order to provide ethical hackers with similar
tools, and to understand the impact and the limitations of artificial agents,
we present in this paper a model that formalizes web hacking tasks for
reinforcement learning agents. Our model, named Agent Web Model, considers web
hacking as a capture-the-flag style challenge, and it defines reinforcement
learning problems at seven different levels of abstraction. We discuss the
complexity of these problems in terms of actions and states an agent has to
deal with, and we show that such a model allows to represent most of the
relevant web vulnerabilities. Aware that the driver of advances in
reinforcement learning is the availability of standardized challenges, we
provide an implementation for the first three abstraction layers, in the hope
that the community would consider these challenges in order to develop
intelligent web hacking agents.
