  Approximation of elliptic PDEs with random diffusion coefficients typically
requires a representation of the diffusion field in terms of a sequence
$y=(y_j)_{j\geq 1}$ of scalar random variables. One may then apply
high-dimensional approximation methods to the solution map $y\mapsto u(y)$.
Although Karhunen-Lo\`eve representations are commonly used, it was recently
shown, in the relevant case of lognormal diffusion fields, that they do not
generally yield optimal approximation rates. Motivated by these results, we
construct wavelet-type representations of stationary Gaussian random fields
defined on bounded domains. The size and localization properties of these
wavelets are studied, and used to obtain polynomial approximation results for
the related elliptic PDE which outperform those achievable when using
Karhunen-Lo\`eve representations. Our construction is based on a periodic
extension of the random field, and the expansion on the domain is then obtained
by simple restriction. This makes the approach easily applicable even when the
computational domain of the PDE has a complicated geometry. In particular, we
apply this construction to the class of Gaussian processes defined by the
family of Mat\'ern covariances.
