  We propose a novel framework for the regularised inversion of deep neural
networks. The framework is based on the authors' recent work on training
feed-forward neural networks without the differentiation of activation
functions. The framework lifts the parameter space into a higher dimensional
space by introducing auxiliary variables, and penalises these variables with
tailored Bregman distances. We propose a family of variational regularisations
based on these Bregman distances, present theoretical results and support their
practical application with numerical examples. In particular, we present the
first convergence result (to the best of our knowledge) for the regularised
inversion of a single-layer perceptron that only assumes that the solution of
the inverse problem is in the range of the regularisation operator, and that
shows that the regularised inverse provably converges to the true inverse if
measurement errors converge to zero.
