  Obtaining sharp Lipschitz constants for feed-forward neural networks is
essential to assess their robustness in the face of perturbations of their
inputs. We derive such constants in the context of a general layered network
model involving compositions of nonexpansive averaged operators and affine
operators. By exploiting this architecture, our analysis finely captures the
interactions between the layers, yielding tighter Lipschitz constants than
those resulting from the product of individual bounds for groups of layers. The
proposed framework is shown to cover in particular many practical instances
encountered in feed-forward neural networks. Our Lipschitz constant estimates
are further improved in the case of structures employing scalar nonlinear
functions, which include standard convolutional networks as special cases.
