  A linear restriction of a function is the same function with its domain
restricted to points on a given line. This paper addresses the problem of
computing a succinct representation for a linear restriction of a
piecewise-linear neural network. This primitive, which we call ExactLine,
allows us to exactly characterize the result of applying the network to all of
the infinitely many points on a line. In particular, ExactLine computes a
partitioning of the given input line segment such that the network is affine on
each partition. We present an efficient algorithm for computing ExactLine for
networks that use ReLU, MaxPool, batch normalization, fully-connected,
convolutional, and other layers, along with several applications. First, we
show how to exactly determine decision boundaries of an ACAS Xu neural network,
providing significantly improved confidence in the results compared to prior
work that sampled finitely many points in the input space. Next, we demonstrate
how to exactly compute integrated gradients, which are commonly used for neural
network attributions, allowing us to show that the prior heuristic-based
methods had relative errors of 25-45% and show that a better sampling method
can achieve higher accuracy with less computation. Finally, we use ExactLine to
empirically falsify the core assumption behind a well-known hypothesis about
adversarial examples, and in the process identify interesting properties of
adversarially-trained networks.
