  We explore connections between covariance representations, Bismut-type
formulas and Stein's method. First, using the theory of closed symmetric forms,
we derive covariance representations for several well-known probability
measures on $\mathbb{R}^d$, $d \geq 1$. When strong gradient bounds are
available, these covariance representations immediately lead to $L^p$-$L^q$
covariance estimates, for all $p \in (1, +\infty)$ and $q = p/(p-1)$. Then, we
revisit the well-known $L^p$-Poincar\'e inequalities ($p \geq 2$) for the
standard Gaussian probability measure on $\mathbb{R}^d$ based on a covariance
representation. Moreover, for the nondegenerate symmetric $\alpha$-stable case,
$\alpha \in (1,2)$, we obtain $L^p$-Poincar\'e and pseudo-Poincar\'e
inequalities, for $p \in (1, \alpha)$, via a detailed analysis of the various
Bismut-type formulas at our disposal. Finally, using the construction of
Stein's kernels by closed forms techniques, we obtain quantitative
high-dimensional CLTs in $1$-Wasserstein distance when the limiting Gaussian
probability measure is anisotropic. The dependence on the parameters is
completely explicit and the rates of convergence are sharp.
