  Motivated by the residual type neural networks (ResNet), this paper studies
optimal control problems constrained by a non-smooth integral equation. Such
non-smooth equations, for instance, arise in the continuous representation of
fractional deep neural networks (DNNs). Here the underlying non-differentiable
function
  is the ReLU or max function. The control enters in a nonlinear and
multiplicative manner and we additionally impose control constraints. Because
of the presence of the non-differentiable mapping, the application of standard
adjoint calculus is excluded. We derive strong stationary conditions by relying
on the limited differentiability properties of the non-smooth map. While
traditional approaches smoothen the non-differentiable function, no such
smoothness is retained in our final strong stationarity system. Thus, this work
also closes a gap which currently exists in continuous neural networks with
ReLU type activation function.
