  A critical issue in approximating solutions of ordinary differential
equations using neural networks is the exact satisfaction of the boundary or
initial conditions. For this purpose, neural forms have been introduced, i.e.,
functional expressions that depend on neural networks which, by design, satisfy
the prescribed conditions exactly. Expanding upon prior progress, the present
work contributes in three distinct aspects. First, it presents a novel
formalism for crafting optimized neural forms. Second, it outlines a method for
establishing an upper bound on the absolute deviation from the exact solution.
Third, it introduces a technique for converting problems with Neumann or Robin
conditions into equivalent problems with parametric Dirichlet conditions. The
proposed optimized neural forms were numerically tested on a set of diverse
problems, encompassing first-order and second-order ordinary differential
equations, as well as first-order systems. Stiff and delay differential
equations were also considered. The obtained solutions were compared against
solutions obtained via Runge-Kutta methods and exact solutions wherever
available. The reported results and analysis verify that in addition to the
exact satisfaction of the boundary or initial conditions, optimized neural
forms provide closed-form solutions of superior interpolation capability and
controllable overall accuracy.
