  Although Neural Differential Equations have shown promise on toy problems
such as MNIST, they have yet to be successfully applied to more challenging
tasks. Inspired by variational methods for image restoration relying on partial
differential equations, we choose to benchmark several forms of Neural DEs and
backpropagation methods on single image super-resolution. The adjoint method
previously proposed for gradient estimation has no theoretical stability
guarantees; we find a practical case where this makes it unusable, and show
that discrete sensitivity analysis has better stability. In our experiments,
differential models match the performance of a state-of-the art
super-resolution model.
