  Intrusion detection systems perform post-compromise detection of security
breaches whenever preventive measures such as firewalls do not avert an attack.
However, these systems raise a vast number of alerts that must be analysed and
triaged by security analysts. This process is largely manual, tedious and
time-consuming. Alert correlation is a technique that tries to reduce the
number of intrusion alerts by aggregating those that are related in some way.
However, the correlation is performed outside the IDS through third-party
systems and tools, after the high volume of alerts has already been raised.
These other third-party systems add to the complexity of security operations.
In this paper, we build on the very researched area of correlation techniques
by developing a novel hierarchical event correlation model that promises to
reduce the number of alerts issued by an Intrusion Detection System. This is
achieved by correlating the events before the IDS classifies them. The proposed
model takes the best of features from similarity and graph-based correlation
techniques to deliver an ensemble capability not possible by either approach
separately. Further, we propose a correlation process for correlation of events
rather than alerts as is the case in current art. We further develop our own
correlation and clustering algorithm which is tailor-made to the correlation
and clustering of network event data. The model is implemented as a proof of
concept with experiments run on the DARPA 99 Intrusion detection set. The
correlation achieved 87 percent data reduction through aggregation, producing
nearly 21000 clusters in about 30 seconds.
