  The method recently introduced in arXiv:2011.10115 realizes a deep neural
network with just a single nonlinear element and delayed feedback. It is
applicable for the description of physically implemented neural networks. In
this work, we present an infinite-dimensional generalization, which allows for
a more rigorous mathematical analysis and a higher flexibility in choosing the
weight functions. Precisely speaking, the weights are described by Lebesgue
integrable functions instead of step functions. We also provide a functional
back-propagation algorithm, which enables gradient descent training of the
weights. In addition, with a slight modification, our concept realizes
recurrent neural networks.
