  We propose a new approach to non-parametric density estimation that is based
on regularizing a Sobolev norm of the density. This method is statistically
consistent, and makes the inductive bias of the model clear and interpretable.
While there is no closed analytic form for the associated kernel, we show that
one can approximate it using sampling. The optimization problem needed to
determine the density is non-convex, and standard gradient methods do not
perform well. However, we show that with an appropriate initialization and
using natural gradients, one can obtain well performing solutions. Finally,
while the approach provides pre-densities (i.e. not necessarily integrating to
1), which prevents the use of log-likelihood for cross validation, we show that
one can instead adapt Fisher divergence based score matching methods for this
task. We evaluate the resulting method on the comprehensive recent anomaly
detection benchmark suite, ADBench, and find that it ranks second best, among
more than 15 algorithms.
