  Machine learning has shown tremendous potential for improving the
capabilities of network traffic analysis applications, often outperforming
simpler rule-based heuristics. However, ML-based solutions remain difficult to
deploy in practice. Many existing approaches only optimize the predictive
performance of their models, overlooking the practical challenges of running
them against network traffic in real time. This is especially problematic in
the domain of traffic analysis, where the efficiency of the serving pipeline is
a critical factor in determining the usability of a model. In this work, we
introduce CATO, a framework that addresses this problem by jointly optimizing
the predictive performance and the associated systems costs of the serving
pipeline. CATO leverages recent advances in multi-objective Bayesian
optimization to efficiently identify Pareto-optimal configurations, and
automatically compiles end-to-end optimized serving pipelines that can be
deployed in real networks. Our evaluations show that compared to popular
feature optimization techniques, CATO can provide up to 3600x lower inference
latency and 3.7x higher zero-loss throughput while simultaneously achieving
better model performance.
