  The multivariate Ornstein-Uhlenbeck process is used in many branches of
science and engineering to describe the regression of a system to its
stationary mean. Here we present an $O(N)$ Bayesian method to estimate the
drift and diffusion matrices of the process from $N$ discrete observations of a
sample path. We use exact likelihoods, expressed in terms of four sufficient
statistic matrices, to derive explicit maximum a posteriori parameter estimates
and their standard errors. We apply the method to the Brownian harmonic
oscillator, a bivariate Ornstein-Uhlenbeck process, to jointly estimate its
mass, damping, and stiffness and to provide Bayesian estimates of the
correlation functions and power spectral densities. We present a Bayesian model
comparison procedure, embodying Ockham's razor, to guide a data-driven choice
between the Kramers and Smoluchowski limits of the oscillator. These provide
novel methods of analyzing the inertial motion of colloidal particles in
optical traps.
