  Convergence properties of Shannon Entropy are studied. In the differential
setting, it is shown that weak convergence of probability measures, or
convergence in distribution, is not enough for convergence of the associated
differential entropies. A general result for the desired differential entropy
convergence is provided, taking into account both compactly and uncompactly
supported densities. Convergence of differential entropy is also characterized
in terms of the Kullback-Liebler discriminant for densities with fairly general
supports, and it is shown that convergence in variation of probability measures
guarantees such convergence under an appropriate boundedness condition on the
densities involved. Results for the discrete setting are also provided,
allowing for infinitely supported probability measures, by taking advantage of
the equivalence between weak convergence and convergence in variation in this
setting.
