  We propose probabilistic representations for inverse Stein operators (i.e.
solutions to Stein equations) under general conditions; in particular we deduce
new simple expressions for the Stein kernel. These representations allow to
deduce uniform and non-uniform Stein factors (i.e. bounds on solutions to Stein
equations) and lead to new covariance identities expressing the covariance
between arbitrary functionals of an arbitrary {univariate} target in terms of a
weighted covariance of the derivatives of the functionals. Our weights are
explicit, easily computable in most cases, and expressed in terms of objects
familiar within the context of Stein's method. Applications of the
Cauchy-Schwarz inequality to these weighted covariance identities lead to sharp
upper and lower covariance bounds and, in particular, weighted Poincar\'e
inequalities. Many examples are given and, in particular, classical variance
bounds due to Klaassen, Brascamp and Lieb or Otto and Menz are corollaries.
Connections with more recent literature are also detailed.
