  We study the problem of computing the preimage of a set under a neural
network with piecewise-affine activation functions. We recall an old result
that the preimage of a polyhedral set is again a union of polyhedral sets and
can be effectively computed. We show several applications of computing the
preimage for analysis and interpretability of neural networks.
