  A frequent and well-founded criticism of the maximum a posteriori (MAP) and
minimum mean squared error (MMSE) estimates of a continuous parameter \gamma
taking values in a differentiable manifold \Gamma is that they are not
invariant to arbitrary ``reparameterizations'' of \Gamma. This paper clarifies
the issues surrounding this problem, by pointing out the difference between
coordinate invariance, which is a sine qua non for a mathematically
well-defined problem, and diffeomorphism invariance, which is a substantial
issue, and then provides a solution. We first show that the presence of a
metric structure on \Gamma can be used to define coordinate-invariant MAP and
MMSE estimates, and we argue that this is the natural way to proceed. We then
discuss the choice of a metric structure on \Gamma. By imposing an invariance
criterion natural within a Bayesian framework, we show that this choice is
essentially unique. It does not necessarily correspond to a choice of
coordinates. In cases of complete prior ignorance, when Jeffreys' prior is
used, the invariant MAP estimate reduces to the maximum likelihood estimate.
The invariant MAP estimate coincides with the minimum message length (MML)
estimate, but no discretization or approximation is used in its derivation.
