  We investigate the asymptotic normality of the posterior distribution in the
discrete setting, when model dimension increases with sample size. We consider
a probability mass function $\theta_0$ on $\mathbbm{N}\setminus \{0\}$ and a
sequence of truncation levels $(k_n)_n$ satisfying $k_n^3\leq n\inf_{i\leq
k_n}\theta_0(i).$ Let $\hat{\theta}$ denote the maximum likelihood estimate of
$(\theta_0(i))_{i\leq k_n}$ and let $\Delta_n(\theta_0)$ denote the
$k_n$-dimensional vector which $i$-th coordinate is defined by \sqrt{n}
(\hat{\theta}_n(i)-\theta_0(i)) for $1\leq i\leq k_n.$ We check that under mild
conditions on $\theta_0$ and on the sequence of prior probabilities on the
$k_n$-dimensional simplices, after centering and rescaling, the variation
distance between the posterior distribution recentered around $\hat{\theta}_n$
and rescaled by $\sqrt{n}$ and the $k_n$-dimensional Gaussian distribution
$\mathcal{N}(\Delta_n(\theta_0),I^{-1}(\theta_0))$ converges in probability to
$0.$ This theorem can be used to prove the asymptotic normality of Bayesian
estimators of Shannon and R\'{e}nyi entropies. The proofs are based on
concentration inequalities for centered and non-centered Chi-square (Pearson)
statistics. The latter allow to establish posterior concentration rates with
respect to Fisher distance rather than with respect to the Hellinger distance
as it is commonplace in non-parametric Bayesian statistics.
