  Modern network performance monitoring toolkits, such as perfSONAR, take a
remarkable number of measurements about the local network environment. To gain
a complete picture of network performance, however, one needs to aggregate data
across a large number of endpoints. The Service Analysis and Network Diagnosis
(SAND) data pipeline collects data from diverse sources and ingests these
measurements into a message bus. The message bus allows the project to send the
data to multiple consumers, including a tape archive, an Elasticsearch
database, and a peer infrastructure at CERN. In this paper, we explain the
architecture and evolution of the SAND data pipeline, the scale of the
resulting dataset, and how it supports a wide variety of network analysis
applications.
