  Recently, Mutual Information (MI) has attracted attention in bounding the
generalization error of Deep Neural Networks (DNNs). However, it is intractable
to accurately estimate the MI in DNNs, thus most previous works have to relax
the MI bound, which in turn weakens the information theoretic explanation for
generalization. To address the limitation, this paper introduces a
probabilistic representation of DNNs for accurately estimating the MI.
Leveraging the proposed MI estimator, we validate the information theoretic
explanation for generalization, and derive a tighter generalization bound than
the state-of-the-art relaxations.
