  Anomaly detection is a method for discovering unusual and suspicious
behavior. In many real-world scenarios, the examined events can be directly
linked to the actions of an adversary, such as attacks on computer networks or
frauds in financial operations. While the defender wants to discover such
malicious behavior, the attacker seeks to accomplish their goal (e.g.,
exfiltrating data) while avoiding the detection. To this end, anomaly detectors
have been used in a game-theoretic framework that captures these goals of a
two-player competition. We extend the existing models to more realistic
settings by (1) allowing both players to have continuous action spaces and by
assuming that (2) the defender cannot perfectly observe the action of the
attacker. We propose two algorithms for solving such games -- a direct
extension of existing algorithms based on discretizing the feature space and
linear programming and the second algorithm based on constrained learning.
Experiments show that both algorithms are applicable for cases with low feature
space dimensions but the learning-based method produces less exploitable
strategies and it is scalable to higher dimensions. Moreover, we use real-world
data to compare our approaches with existing classifiers in a data-exfiltration
scenario via the DNS channel. The results show that our models are
significantly less exploitable by an informed attacker.
