  Diffusion models are generative models that have recently demonstrated
impressive performances in terms of sampling quality and density estimation in
high dimensions. They rely on a forward continuous diffusion process and a
backward continuous denoising process, which can be described by a
time-dependent vector field and is used as a generative model. In the original
formulation of the diffusion model, this vector field is assumed to be the
score function (i.e. it is the gradient of the log-probability at a given time
in the diffusion process). Curiously, on the practical side, most studies on
diffusion models implement this vector field as a neural network function and
do not constrain it be the gradient of some energy function (that is, most
studies do not constrain the vector field to be conservative). Even though some
studies investigated empirically whether such a constraint will lead to a
performance gain, they lead to contradicting results and failed to provide
analytical results. Here, we provide three analytical results regarding the
extent of the modeling freedom of this vector field. {Firstly, we propose a
novel decomposition of vector fields into a conservative component and an
orthogonal component which satisfies a given (gauge) freedom. Secondly, from
this orthogonal decomposition, we show that exact density estimation and exact
sampling is achieved when the conservative component is exactly equals to the
true score and therefore conservativity is neither necessary nor sufficient to
obtain exact density estimation and exact sampling. Finally, we show that when
it comes to inferring local information of the data manifold, constraining the
vector field to be conservative is desirable.
