  This study focuses on a Wasserstein-type gradient flow, which represents an
optimization process of a continuous model of a Deep Neural Network (DNN).
First, we establish the existence of a minimizer for an average loss of the
model under $L^2$-regularization. Subsequently, we show the existence of a
curve of maximal slope of the loss. Our main result is the convergence of flow
to a critical point of the loss as time goes to infinity. An essential aspect
of proving this result involves the establishment of the \L{}ojasiewicz--Simon
gradient inequality for the loss. We derive this inequality by assuming the
analyticity of NNs and loss functions. Our proofs offer a new approach for
analyzing the asymptotic behavior of Wasserstein-type gradient flows for
nonconvex functionals.
