  The posterior covariance matrix W defined by the log-likelihood of each
observation plays important roles both in the sensitivity analysis and
frequentist evaluation of the Bayesian estimators. This study is focused on the
matrix W and its principal space; we term the latter as an essential subspace.
Projections to the essential subspace realize dimensional reduction in the
sensitivity analysis and frequentist evaluation. A key tool for treating
frequentist properties is the recently proposed Bayesian infinitesimal
jackknife approximation(Giordano and Broderick (2023)). The matrix W can be
interpreted as a reproducing kernel and is denoted as W-kernel. Using W-kernel,
the essential subspace is expressed as a principal space given by the kernel
principal component analysis. A relation to the Fisher kernel and neural
tangent kernel is established, which elucidates the connection to the classical
asymptotic theory. We also discuss a type of Bayesian-frequentist duality,
naturally appeared from the kernel framework. Two applications are discussed:
the selection of a representative set of observations and dimensional reduction
in the approximate bootstrap. In the former, incomplete Cholesky decomposition
is introduced as an efficient method for computing the essential subspace. In
the latter, different implementations of the approximate bootstrap for
posterior means are compared.
