  Mutual information $I(X;Y)$ is a useful definition in information theory to
estimate how much information the random variable $Y$ holds about the random
variable $X$. One way to define the mutual information is by comparing the
joint distribution of $X$ and $Y$ with the product of the marginals through the
KL-divergence. If the two distributions are close to each other there will be
almost no leakage of $X$ from $Y$ since the two variables are close to being
independent. In the discrete setting the mutual information has the nice
interpretation of how many bits $Y$ reveals about $X$ and if $I(X;Y)=H(X)$ (the
Shannon entropy of $X$) then $X$ is completely revealed. However, in the
continuous case we do not have the same reasoning. For instance the mutual
information can be infinite in the continuous case. This fact enables us to try
different metrics or divergences to define the mutual information. In this
paper, we are evaluating different metrics or divergences such as
Kullback-Liebler (KL) divergence, Wasserstein distance, Jensen-Shannon
divergence and total variation distance to form alternatives to the mutual
information in the continuous case. We deploy different methods to estimate or
bound these metrics and divergences and evaluate their performances.
