  Bayesian networks (BNs) are graphical \emph{first-order} probabilistic models
that allow for a compact representation of large probability distributions, and
for efficient inference, both exact and approximate. We introduce a
\emph{higher-order} programming language -- in the idealized form of a
$\lambda$-calculus -- which we prove \emph{sound and complete} w.r.t. BNs: each
BN can be encoded as a term, and conversely each (possibly higher-order and
recursive) program of ground type \emph{compiles} into a BN. The language
allows for the specification of recursive probability models and hierarchical
structures. Moreover, we provide a \emph{compositional} and \emph{cost-aware}
semantics which is based on factors, the standard mathematical tool used in
Bayesian inference. Our results rely on advanced techniques rooted into linear
logic, intersection types, rewriting theory, and Girard's geometry of
interaction, which are here combined in a novel way.
