  Software Defined Networking (SDN) enables flexible and scalable network
control and management. However, it also introduces new vulnerabilities that
can be exploited by attackers. In particular, low-rate and slow or stealthy
Denial-of-Service (DoS) attacks are recently attracting attention from
researchers because of their detection challenges. In this paper, we propose a
novel machine learning based defense framework named Q-MIND, to effectively
detect and mitigate stealthy DoS attacks in SDN-based networks. We first
analyze the adversary model of stealthy DoS attacks, the related
vulnerabilities in SDN-based networks and the key characteristics of stealthy
DoS attacks. Next, we describe and analyze an anomaly detection system that
uses a Reinforcement Learning-based approach based on Q-Learning in order to
maximize its detection performance. Finally, we outline the complete Q-MIND
defense framework that incorporates the optimal policy derived from the
Q-Learning agent to efficiently defeat stealthy DoS attacks in SDN-based
networks. An extensive comparison of the Q-MIND framework and currently
existing methods shows that significant improvements in attack detection and
mitigation performance are obtained by Q-MIND.
