  We present a new perturbative approach to "constrained Ly$\alpha$ radiative
transfer'" (RT) through the circum- and inter-galactic medium (CGM and IGM). We
constrain the HI content and kinematics of the CGM and IGM in a physically
motivated model, using the galaxy-Ly$\alpha$ forest clustering data from
spectroscopic galaxy surveys in QSO fields at $z\sim2-3$. This enables us to
quantify the impact of the CGM/IGM on Ly$\alpha$ emission in an observationally
constrained, realistic cosmological environment. Our model predicts that the
CGM and IGM at these redshifts transmit $\approx80~\%$ of Ly$\alpha$ photons
after having escaped from galaxies. This implies that while the inter-stellar
medium primarily regulates Ly$\alpha$ escape, the CGM has a non-negligible
impact on the observed Ly$\alpha$ line properties and the inferred Ly$\alpha$
escape fraction, even at $z\sim 2-3$. Ly$\alpha$ scattering in the CGM and IGM
further introduces an environmental dependence in the (apparent) Ly$\alpha$
escape fraction, and the observed population of Ly$\alpha$ emitting galaxies:
the CGM/IGM more strongly suppresses direct Ly$\alpha$ emission from galaxies
in overdense regions in the Universe, and redistributes this emission into
brighter Ly$\alpha$ haloes. The resulting mean surface brightness profile of
the Ly$\alpha$ haloes is generally found to be a power-law $\propto r^{-2.4}$.
Although our model still contains arbitrariness, our results demonstrate how
(integral field) spectroscopic surveys of galaxies in QSO fields constrain
circumgalactic Ly$\alpha$ RT, and we discuss the potential of these models for
studying CGM physics and cosmology.
