  In the Bayes paradigm and for a given loss function, we propose the
construction of a new type of posterior distributions, that extends the
classical Bayes one, for estimating the law of an $n$-sample. The loss
functions we have in mind are based on the total variation and Hellinger
distances as well as some $\mathbb{L}_{j}$-ones. We prove that, with a
probability close to one, this new posterior distribution concentrates its mass
in a neighbourhood of the law of the data, for the chosen loss function,
provided that this law belongs to the support of the prior or, at least, lies
close enough to it. We therefore establish that the new posterior distribution
enjoys some robustness properties with respect to a possible misspecification
of the prior, or more precisely, its support. For the total variation and
squared Hellinger losses, we also show that the posterior distribution keeps
its concentration properties when the data are only independent, hence not
necessarily i.i.d., provided that most of their marginals or the average of
these are close enough to some probability distribution around which the prior
puts enough mass. The posterior distribution is therefore also stable with
respect to the equidistribution assumption. We illustrate these results by
several applications. We consider the problems of estimating a location
parameter or both the location and the scale of a density in a nonparametric
framework. Finally, we also tackle the problem of estimating a density, with
the squared Hellinger loss, in a high-dimensional parametric model under some
sparsity conditions. The results established in this paper are non-asymptotic
and provide, as much as possible, explicit constants.
