  We study in detail the class of all polynomial Stein operators (Stein
operators with polynomial coefficients) associated to the standard Gaussian
distribution via a connection between Stein's method and noncommutative
algebra. We provide a complete description of this class of Stein operators by
(i) identifying it as a vector space over $\mathbb{R}$ with an explicit given
basis and (ii) by showing that this class is a principal right ideal of the
first Weyl algebra generated by the classical Gaussian Stein operator $\partial
-x$, with $\partial$ denoting the usual differential operator. We also study
the characterising property of polynomial Stein operators for the standard
Gaussian distribution, and give examples of general classes of polynomial Stein
operators that are characterising, as well as classes that are not
characterising unless additional distributional assumptions are made. By
appealing to a standard property of Weyl algebras, we shown that the
non-characterising property possessed by a wide class of polynomial Stein
operators for the standard Gaussian distribution is a consequence of a result
that is perhaps surprising from a probabilistic perspective: the intersection
between the class of polynomial Stein operators for any two target
distributions with holonomic characteristic functions is non-trivial.
