  We investigate the approximation of high-dimensional target measures as
low-dimensional updates of a dominating reference measure. This approximation
class replaces the associated density with the composition of: (i) a feature
map that identifies the leading principal components or features of the target
measure, relative to the reference, and (ii) a low-dimensional profile
function. When the reference measure satisfies a subspace $\phi$-Sobolev
inequality, we construct a computationally tractable approximation that yields
certifiable error guarantees with respect to the Amari $\alpha$-divergences.
Our construction proceeds in two stages. First, for any feature map and any
$\alpha$-divergence, we obtain an analytical expression for the optimal profile
function. Second, for linear feature maps, the principal features are obtained
from eigenvectors of a matrix involving gradients of the log-density. Neither
step requires explicit access to normalizing constants. Notably, by leveraging
the $\phi$-Sobolev inequalities, we demonstrate that these features universally
certify approximation errors across the range of $\alpha$-divergences $\alpha
\in (0,1]$. We then propose an application to Bayesian inverse problems and
provide an analogous construction with approximation guarantees that hold in
expectation over the data. We conclude with an extension of the proposed
dimension reduction strategy to nonlinear feature maps.
