  Solutions to differential equations are of significant scientific and
engineering relevance. Physics-Informed Neural Networks (PINNs) have emerged as
a promising method for solving differential equations, but they lack a
theoretical justification for the use of any particular loss function. This
work presents Differential Equation GAN (DEQGAN), a novel method for solving
differential equations using generative adversarial networks to "learn the loss
function" for optimizing the neural network. Presenting results on a suite of
twelve ordinary and partial differential equations, including the nonlinear
Burgers', Allen-Cahn, Hamilton, and modified Einstein's gravity equations, we
show that DEQGAN can obtain multiple orders of magnitude lower mean squared
errors than PINNs that use $L_2$, $L_1$, and Huber loss functions. We also show
that DEQGAN achieves solution accuracies that are competitive with popular
numerical methods. Finally, we present two methods to improve the robustness of
DEQGAN to different hyperparameter settings.
