  NetMemex explores efficient network traffic archival without any loss of
information. Unlike NetFlow-like aggregation, NetMemex allows retrieving the
entire packet data including full payload, which makes it useful in forensic
analysis, networked and distributed system research, and network
administration. Different from packet trace dumps, NetMemex performs
sophisticated data compression for small storage space use and optimizes the
data layout for fast query processing. NetMemex takes advantage of high-speed
random access of flash drives and inexpensive storage space of hard disk
drives. These efforts lead to a cost-effective yet high-performance full
traffic archival system. We demonstrate that NetMemex can record full-fidelity
traffic at near-Gbps rates using a single commodity machine, handling common
queries at up to 90.1 K queries/second, at a low storage cost comparable to
conventional hard disk-only traffic archival solutions.
