  Huge datasets in cyber security, such as network traffic logs, can be
analyzed using machine learning and data mining methods. However, the amount of
collected data is increasing, which makes analysis more difficult. Many machine
learning methods have not been designed for big datasets, and consequently are
slow and difficult to understand. We address the issue of efficient network
traffic classification by creating an intrusion detection framework that
applies dimensionality reduction and conjunctive rule extraction. The system
can perform unsupervised anomaly detection and use this information to create
conjunctive rules that classify huge amounts of traffic in real time. We test
the implemented system with the widely used KDD Cup 99 dataset and real-world
network logs to confirm that the performance is satisfactory. This system is
transparent and does not work like a black box, making it intuitive for domain
experts, such as network administrators.
