  The problem of estimating an arbitrary random vector from its observation
corrupted by additive white Gaussian noise, where the cost function is taken to
be the Minimum Mean $p$-th Error (MMPE), is considered. The classical Minimum
Mean Square Error (MMSE) is a special case of the MMPE. Several bounds,
properties and applications of the MMPE are derived and discussed. The optimal
MMPE estimator is found for Gaussian and binary input distributions. Properties
of the MMPE as a function of the input distribution, SNR and order $p$ are
derived. In particular, it is shown that the MMPE is a continuous function of
$p$ and SNR. These results are possible in view of interpolation and change of
measure bounds on the MMPE. The `Single-Crossing-Point Property' (SCPP) that
bounds the MMSE for all SNR values {\it above} a certain value, at which the
MMSE is known, together with the I-MMSE relationship is a powerful tool in
deriving converse proofs in information theory. By studying the notion of
conditional MMPE, a unifying proof (i.e., for any $p$) of the SCPP is shown. A
complementary bound to the SCPP is then shown, which bounds the MMPE for all
SNR values {\it below} a certain value, at which the MMPE is known. As a first
application of the MMPE, a bound on the conditional differential entropy in
terms of the MMPE is provided, which then yields a generalization of the
Ozarow-Wyner lower bound on the mutual information achieved by a discrete input
on a Gaussian noise channel. As a second application, the MMPE is shown to
improve on previous characterizations of the phase transition phenomenon that
manifests, in the limit as the length of the capacity achieving code goes to
infinity, as a discontinuity of the MMSE as a function of SNR. As a final
application, the MMPE is used to show bounds on the second derivative of mutual
information, that tighten previously known bounds.
