  Gaussian approximations are routinely employed in Bayesian statistics to ease
inference when the target posterior is intractable. Although these
approximations are asymptotically justified by Bernstein-von Mises type
results, in practice the expected Gaussian behavior may poorly represent the
shape of the posterior, thus affecting approximation accuracy. Motivated by
these considerations, we derive an improved class of closed-form approximations
of posterior distributions which arise from a new treatment of a third-order
version of the Laplace method yielding approximations in a tractable family of
skew-symmetric distributions. Under general assumptions which account for
misspecified models and non-i.i.d. settings, this family of approximations is
shown to have a total variation distance from the target posterior whose rate
of convergence improves by at least one order of magnitude the one established
by the classical Bernstein-von Mises theorem. Specializing this result to the
case of regular parametric models shows that the same improvement in
approximation accuracy can be also derived for polynomially bounded posterior
functionals. Unlike other higher-order approximations, our results prove that
it is possible to derive closed-form and valid densities which are expected to
provide, in practice, a more accurate, yet similarly-tractable, alternative to
Gaussian approximations of the target posterior, while inheriting its limiting
frequentist properties. We strengthen such arguments by developing a practical
skew-modal approximation for both joint and marginal posteriors that achieves
the same theoretical guarantees of its theoretical counterpart by replacing the
unknown model parameters with the corresponding MAP estimate. Empirical studies
confirm that our theoretical results closely match the remarkable performance
observed in practice, even in finite, possibly small, sample regimes.
