  The Kullback-Leibler (KL) divergence is a fundamental equation of information
theory that quantifies the proximity of two probability distributions. Although
difficult to understand by examining the equation, an intuition and
understanding of the KL divergence arises from its intimate relationship with
likelihood theory. We discuss how KL divergence arises from likelihood theory
in an attempt to provide some intuition and reserve a rigorous (but rather
simple) derivation for the appendix. Finally, we comment on recent applications
of KL divergence in the neural coding literature and highlight its natural
application.
