  The Internet, as a global system of interconnected networks, carries an
extensive array of information resources and services. Key requirements include
good quality-of-service and protection of the infrastructure from nefarious
activity (e.g. distributed denial of service--DDoS--attacks). Network
monitoring is essential to network engineering, capacity planning and
prevention / mitigation of threats. We develop an open source architecture,
AMON (All-packet MONitor), for online monitoring and analysis of multi-gigabit
network streams. It leverages the high-performance packet monitor PF RING and
is readily deployable on commodity hardware. AMON examines all packets,
partitions traffic into sub-streams by using rapid hashing and computes certain
real-time data products. The resulting data structures provide views of the
intensity and connectivity structure of network traffic at the time-scale of
routing. The proposed integrated framework includes modules for the
identification of heavy-hitters as well as for visualization and statistical
detection at the time-of-onset of high impact events such as DDoS. This allows
operators to quickly visualize and diagnose attacks, and limit offline and time
consuming post-mortem analysis. We demonstrate our system in the context of
real-world attack incidents, and validate it against state-of-the-art
alternatives. AMON has been deployed and is currently processing 10Gbps+ live
Internet traffic at Merit Network. It is extensible and allows the addition of
further statistical and filtering modules for real-time forensics.
