  We introduce MESSY estimation, a Maximum-Entropy based Stochastic and
Symbolic densitY estimation method. The proposed approach recovers probability
density functions symbolically from samples using moments of a Gradient flow in
which the ansatz serves as the driving force. In particular, we construct a
gradient-based drift-diffusion process that connects samples of the unknown
distribution function to a guess symbolic expression. We then show that when
the guess distribution has the maximum entropy form, the parameters of this
distribution can be found efficiently by solving a linear system of equations
constructed using the moments of the provided samples. Furthermore, we use
Symbolic regression to explore the space of smooth functions and find optimal
basis functions for the exponent of the maximum entropy functional leading to
good conditioning. The cost of the proposed method for each set of selected
basis functions is linear with the number of samples and quadratic with the
number of basis functions. However, the underlying acceptance/rejection
procedure for finding optimal and well-conditioned bases adds to the
computational cost. We validate the proposed MESSY estimation method against
other benchmark methods for the case of a bi-modal and a discontinuous density,
as well as a density at the limit of physical realizability. We find that the
addition of a symbolic search for basis functions improves the accuracy of the
estimation at a reasonable additional computational cost. Our results suggest
that the proposed method outperforms existing density recovery methods in the
limit of a small to moderate number of samples by providing a low-bias and
tractable symbolic description of the unknown density at a reasonable
computational cost.
