  We present ST2, an end-to-end solution to analyze distributed dataflows in an
online setting. It is powered by Timely Dataflow, a low-latency, distributed
data-parallel dataflow computational framework, and expands on its predecessor
SnailTrail 1, a system to run online critical path analysis on program activity
graphs derived from dataflow execution traces. ST2 connects to a running Timely
computation, creates the program activity graph representation, and runs
multiple analyses on top of it. Analyses include aggregate metrics, progress
and temporal invariant checking, and graph pattern matching. Through a
command-line interface and a real-time dashboard, users are able to interact
with and visualize ST2's analysis results.
  For ST2's implementation, we discuss Differential Dataflow, a framework that
uses differential computation to incrementalize even complex relational
dataflow operators, as an alternative to Timely Dataflow, but ultimately settle
on using Timely. In our performance evaluations, we are able to show that ST2
is able to comfortably keep up with common streaming computations in offline
and online settings, even exceeding SnailTrail 1's performance. We also
showcase and evaluate ST2 from a functional standpoint in a case study. Using
the dashboard to profile a faulty source computation, we manage to successfully
detect the issues' root cause. We argue that ST2 is an extendable system that
paves the way for users to debug, monitor, and optimize online distributed
dataflows.
