  Bernstein's condition is a key assumption that guarantees fast rates in
machine learning. For example, the Gibbs algorithm with prior $\pi$ has an
excess risk in $O(d_{\pi}/n)$, as opposed to the standard
$O(\sqrt{d_{\pi}/n})$, where $n$ denotes the number of observations and
$d_{\pi}$ is a complexity parameter which depends on the prior $\pi$. In this
paper, we examine the Gibbs algorithm in the context of meta-learning, i.e.,
when learning the prior $\pi$ from $T$ tasks (with $n$ observations each)
generated by a meta distribution. Our main result is that Bernstein's condition
always holds at the meta level, regardless of its validity at the observation
level. This implies that the additional cost to learn the Gibbs prior $\pi$,
which will reduce the term $d_\pi$ across tasks, is in $O(1/T)$, instead of the
expected $O(1/\sqrt{T})$. We further illustrate how this result improves on
standard rates in three different settings: discrete priors, Gaussian priors
and mixture of Gaussians priors.
