  The field of Optimal Control under Partial Differential Equations (PDE)
constraints is rapidly changing under the influence of Deep Learning and the
accompanying automatic differentiation libraries. Novel techniques like
Physics-Informed Neural Networks (PINNs) and Differentiable Programming (DP)
are to be contrasted with established numerical schemes like Direct-Adjoint
Looping (DAL). We present a comprehensive comparison of DAL, PINN, and DP using
a general-purpose mesh-free differentiable PDE solver based on Radial Basis
Functions. Under Laplace and Navier-Stokes equations, we found DP to be
extremely effective as it produces the most accurate gradients; thriving even
when DAL fails and PINNs struggle. Additionally, we provide a detailed
benchmark highlighting the limited conditions under which any of those methods
can be efficiently used. Our work provides a guide to Optimal Control
practitioners and connects them further to the Deep Learning community.
