  We consider the problem of security-aware planning in an unknown stochastic
environment, in the presence of attacks on control signals (i.e., actuators) of
the robot. We model the attacker as an agent who has the full knowledge of the
controller as well as the employed intrusion-detection system and who wants to
prevent the controller from performing tasks while staying stealthy. We
formulate the problem as a stochastic game between the attacker and the
controller and present an approach to express the objective of such an agent
and the controller as a combined linear temporal logic (LTL) formula. We then
show that the planning problem, described formally as the problem of satisfying
an LTL formula in a stochastic game, can be solved via model-free reinforcement
learning when the environment is completely unknown. Finally, we illustrate and
evaluate our methods on two robotic planning case studies.
