  We consider the problem of reducing the dimensions of parameters and data in
non-Gaussian Bayesian inference problems. Our goal is to identify an "informed"
subspace of the parameters and an "informative" subspace of the data so that a
high-dimensional inference problem can be approximately reformulated in
low-to-moderate dimensions, thereby improving the computational efficiency of
many inference techniques. To do so, we exploit gradient evaluations of the
log-likelihood function. Furthermore, we use an information-theoretic analysis
to derive a bound on the posterior error due to parameter and data dimension
reduction. This bound relies on logarithmic Sobolev inequalities, and it
reveals the appropriate dimensions of the reduced variables. We compare our
method with classical dimension reduction techniques, such as principal
component analysis and canonical correlation analysis, on applications ranging
from mechanics to image processing.
