  Bayesian networks (BNs) are a foundational model in machine learning and
causal inference. Their graphical structure can handle high-dimensional
problems, divide them into a sparse collection of smaller ones, underlies Judea
Pearl's causality, and determines their explainability and interpretability.
Despite their popularity, there are almost no resources in the literature on
how to compute Shannon's entropy and the Kullback-Leibler (KL) divergence for
BNs under their most common distributional assumptions. In this paper, we
provide computationally efficient algorithms for both by leveraging BNs'
graphical structure, and we illustrate them with a complete set of numerical
examples. In the process, we show it is possible to reduce the computational
complexity of KL from cubic to quadratic for Gaussian BNs.
