  This paper begins with a description of methods for estimating image
probability density functions that reflects the observation that such data is
usually constrained to lie in restricted regions of the high-dimensional image
space-not every pattern of pixels is an image. It is common to say that images
lie on a lower-dimensional manifold in the high-dimensional space. However, it
is not the case that all points on the manifold have an equal probability of
being images. Images are unevenly distributed on the manifold, and our task is
to devise ways to model this distribution as a probability distribution. We
therefore consider popular generative models. For our purposes,
generative/probabilistic models should have the properties of 1) sample
generation: the possibility to sample from this distribution with the modelled
density function, and 2) probability computation: given a previously unseen
sample from the dataset of interest, one should be able to compute its
probability, at least up to a normalising constant. To this end, we investigate
the use of methods such as normalising flow and diffusion models. We then show
how semantic interpretations are used to describe points on the manifold. To
achieve this, we consider an emergent language framework that uses variational
encoders for a disentangled representation of points that reside on a given
manifold. Trajectories between points on a manifold can then be described as
evolving semantic descriptions. We also show that such probabilistic
descriptions (bounded) can be used to improve semantic consistency by
constructing defences against adversarial attacks. We evaluate our methods with
improved semantic robustness and OoD detection capability, explainable and
editable semantic interpolation, and improved classification accuracy under
patch attacks. We also discuss the limitation in diffusion models.
