  Data-driven models for stellar spectra which depend on stellar labels suffer
from label systematics which decrease model performance: the "stellar labels
gap". To close the stellar labels gap, we present a stellar label independent
model for $\textit{Gaia}$ BP/RP (XP) spectra. We develop a novel implementation
of a variational auto-encoder; a $\textit{scatter}$ VAE, which learns to
generate an XP spectrum and intrinsic scatter without relying on stellar
labels. We demonstrate that our model achieves competitive XP spectra
reconstructions in comparison to stellar label dependent models. We find that
our model learns stellar properties directly from the data itself. We then
apply our model to XP/APOGEE giant stars to study the [$\alpha$/M] information
in $\textit{Gaia}$ XP. We provide strong evidence that the XP spectra contain
meaningful [$\alpha$/M] information by demonstrating that our model learns the
$\alpha$-bimodality $\textit{without relying on stellar label correlations}$,
for stars with $T_{\rm eff} <$ 5000 K. We publicly release our trained model,
codebase and data. Importantly, our stellar label independent model can be
implemented for any/all XP spectra because our model performance scales with
training object density, not training label density.
