  We prove a Bernstein-type bound for the difference between the average of
negative log-likelihoods of independent discrete random variables and the
Shannon entropy, both defined on a countably infinite alphabet. The result
holds for the class of discrete random variables with tails lighter than or on
the same order of a discrete power-law distribution. Most commonly-used
discrete distributions such as the Poisson distribution, the negative binomial
distribution, and the power-law distribution itself belong to this class. The
bound is effective in the sense that we provide a method to compute the
constants in it.
