Q: Computing confidence bounds on entropy given an empirical sample from a multinomial distribution? Say we have a multi-state distribution with a finite number of states $i$ where the probability of observing state $i$ in a sample is $p_i$. Suppose we take $N$ samples from this distribution and we get empirical estimates $\hat{p}_i$ for the $p_i$. Now we want to compute an upper confidence limit for the entropy of the distribution, say the 0.95 quantile for entropy. I've read about confidence intervals for binomial distributions but this is multinomial and not only that we want the confidence bound on overall entropy, not the individual confidence bounds on the $p_i$.
