  We present new M-estimators of the mean and variance of real valued random
variables, based on PAC-Bayes bounds. We analyze the non-asymptotic minimax
properties of the deviations of those estimators for sample distributions
having either a bounded variance or a bounded variance and a bounded kurtosis.
Under those weak hypotheses, allowing for heavy-tailed distributions, we show
that the worst case deviations of the empirical mean are suboptimal. We prove
indeed that for any confidence level, there is some M-estimator whose
deviations are of the same order as the deviations of the empirical mean of a
Gaussian statistical sample, even when the statistical sample is instead
heavy-tailed. Experiments reveal that these new estimators perform even better
than predicted by our bounds, showing deviation quantile functions uniformly
lower at all probability levels than the empirical mean for non Gaussian sample
distributions as simple as the mixture of two Gaussian measures.
