  Differential Equation (DE) is a commonly used modeling method in various
scientific subjects such as finance and biology. The parameters in DE models
often have interesting scientific interpretations, but their values are often
unknown and need to be estimated from the measurements of the DE. In this work,
we propose a Bayesian inference framework to solve the problem of estimating
the parameters of the DE model, from the given noisy and scarce observations of
the solution only. A key issue in this problem is to robustly estimate the
derivatives of a function from noisy observations of only the function values
at given location points, under the assumption of a physical model in the form
of differential equation governing the function and its derivatives. To address
the key issue, we use the Gaussian Process Regression with Constraint (GPRC)
method which jointly model the solution, the derivatives, and the parametric
differential equation, to estimate the solution and its derivatives. For
nonlinear differential equations, a Picard-iteration-like approximation of
linearization method is used so that the GPRC can be still iteratively
applicable. A new potential which combines the data and equation information,
is proposed and used in the likelihood for our inference. With numerical
examples, we illustrate that the proposed method has competitive performance
against existing approaches for estimating the unknown parameters in DEs.
