  Recently, several methods have been proposed for estimating the mutual
information from sample data using deep neural networks. These estimators ar
referred to as neural mutual information estimation (NMIE)s. NMIEs differ from
other approaches as they are data-driven estimators. As such, they have the
potential to perform well on a large class of capacity problems. In order to
test the performance across various NMIEs, it is desirable to establish a
benchmark encompassing the different challenges of capacity estimation. This is
the objective of this paper. In particular, we consider three scenarios for
benchmarking:i the classic AWGN channel, ii channels continuous inputs optical
intensity and peak-power constrained AWGN channel iii channels with a discrete
output, i.e., Poisson channel. We also consider the extension to the
multi-terminal case with iv the AWGN and optical MAC models. We argue that
benchmarking a certain NMIE across these four scenarios provides a substantive
test of performance. In this paper we study the performance of mutual
information neural estimator (MINE), smoothed mutual information lower-bound
estimator (SMILE), and directed information neural estimator (DINE). and
provide insights on the performance of other methods as well. To summarize our
benchmarking results, MINE provides the most reliable performance.
