  This paper explores Bayesian lower bounds on the minimum mean squared error
(MMSE) that belong to the Ziv-Zakai (ZZ) family. The ZZ technique relies on
connecting the bound to an M-ary hypothesis testing problem. Three versions of
the ZZ bound (ZZB) exist: the first relies on the so-called valley-filling
function (VFF), the second omits the VFF, and the third, i.e., the single-point
ZZB (SZZB), uses a single point maximization. The first part of this paper
provides the most general version of the bounds. First, it is shown that these
bounds hold without any assumption on the distribution of the estimand. Second,
the SZZB bound is extended to an M-ary setting and a version of it for the
multivariate case is provided. In the second part, general properties of the
bounds are provided. First, it is shown that all the bounds tensorize. Second,
a complete characterization of the high-noise asymptotic is provided, which is
used to argue about the tightness of the bounds. Third, the low-noise
asymptotic is provided for mixed-input distributions and Gaussian additive
noise channels. Specifically, in the low-noise, it is shown that the SZZB is
not always tight. In the third part, the tightness of the bounds is evaluated.
First, it is shown that in the low-noise regime the ZZB bound without the VFF
is tight for mixed-input distributions and Gaussian additive noise channels.
Second, for discrete inputs, the ZZB with the VFF is shown to be always
sub-optimal, and equal to zero without the VFF. Third, unlike for the ZZB, an
example is shown for which the SZZB is tight to the MMSE for discrete inputs.
Fourth, sufficient and necessary conditions for the tightness of the bounds are
provided. Finally, some examples are shown in which the bounds in the ZZ family
outperform other well-known Bayesian bounds, i.e., the Cram\'er-Rao bound and
the maximum entropy bound.
