  The connection between Residual Neural Networks (ResNets) and continuous-time
control systems (known as NeurODEs) has led to a mathematical analysis of
neural networks which has provided interesting results of both theoretical and
practical significance. However, by construction, NeurODEs have been limited to
describing constant-width layers, making them unsuitable for modeling deep
learning architectures with layers of variable width. In this paper, we propose
a continuous-time Autoencoder, which we call AutoencODE, based on a
modification of the controlled field that drives the dynamics. This adaptation
enables the extension of the mean-field control framework originally devised
for conventional NeurODEs. In this setting, we tackle the case of low Tikhonov
regularization, resulting in potentially non-convex cost landscapes. While the
global results obtained for high Tikhonov regularization may not hold globally,
we show that many of them can be recovered in regions where the loss function
is locally convex. Inspired by our theoretical findings, we develop a training
method tailored to this specific type of Autoencoders with residual
connections, and we validate our approach through numerical experiments
conducted on various examples.
