  The Fisher Information Matrix (FIM) has been the standard approximation to
the accuracy of parameter estimation on gravitational-wave signals from merging
compact binaries due to its ease-of-use and rapid computation time. While the
theoretical failings of this method, such as the signal-to-noise ratio (SNR)
limit on the validity of the lowest-order expansion and the difficulty of using
non-Gaussian priors, are well understood, the practical effectiveness compared
to a real parameter estimation technique (e.g. Markov-chain Monte Carlo)
remains an open question. We present a direct comparison between the FIM error
estimates and the Bayesian probability density functions produced by the
parameter estimation code lalinference_mcmc. In addition to the low-SNR issues
usually considered, we find that the FIM can greatly overestimate the
uncertainty in parameter estimation achievable by the MCMC. This was found to
be a systematic effect for systems composed of binary black holes, with the
disagreement increasing with total mass. In some cases, the MCMC search
returned standard deviations on the marginalized posteriors that were smaller
by several orders of magnitude than the FIM estimates. We conclude that the
predictions of the FIM do not represent the capabilities of real
gravitational-wave parameter estimation.
