  We introduce the Optimizing a Discrete Loss (ODIL) framework for the
numerical solution of Partial Differential Equations (PDE) using machine
learning tools. The framework formulates numerical methods as a minimization of
discrete residuals that are solved using gradient descent and Newton's methods.
We demonstrate the value of this approach on equations that may have missing
parameters or where no sufficient data is available to form a well-posed
initial-value problem. The framework is presented for mesh based
discretizations of PDEs and inherits their accuracy, convergence, and
conservation properties. It preserves the sparsity of the solutions and is
readily applicable to inverse and ill-posed problems. It is applied to
PDE-constrained optimization, optical flow, system identification, and data
assimilation using gradient descent algorithms including those often deployed
in machine learning. We compare ODIL with related approach that represents the
solution with neural networks. We compare the two methodologies and demonstrate
advantages of ODIL that include significantly higher convergence rates and
several orders of magnitude lower computational cost. We evaluate the method on
various linear and nonlinear partial differential equations including the
Navier-Stokes equations for flow reconstruction problems.
