  Throughout many fields, practitioners often rely on differential equations to
model systems. Yet, for many applications, the theoretical derivation of such
equations and/or accurate resolution of their solutions may be intractable.
Instead, recently developed methods, including those based on parameter
estimation, operator subset selection, and neural networks, allow for the
data-driven discovery of both ordinary and partial differential equations
(PDEs), on a spectrum of interpretability. The success of these strategies is
often contingent upon the correct identification of representative equations
from noisy observations of state variables and, as importantly and intertwined
with that, the mathematical strategies utilized to enforce those equations.
Specifically, the latter has been commonly addressed via unconstrained
optimization strategies. Representing the PDE as a neural network, we propose
to discover the PDE by solving a constrained optimization problem and using an
intermediate state representation similar to a Physics-Informed Neural Network
(PINN). The objective function of this constrained optimization problem
promotes matching the data, while the constraints require that the PDE is
satisfied at several spatial collocation points. We present a penalty method
and a widely used trust-region barrier method to solve this constrained
optimization problem, and we compare these methods on numerical examples. Our
results on the Burgers' and the Korteweg-De Vreis equations demonstrate that
the latter constrained method outperforms the penalty method, particularly for
higher noise levels or fewer collocation points. For both methods, we solve
these discovered neural network PDEs with classical methods, such as finite
difference methods, as opposed to PINNs-type methods relying on automatic
differentiation. We briefly highlight other small, yet crucial, implementation
details.
