  Gravitational waves from merging compact objects encode direct information
about the luminosity distance to the binary. When paired with a redshift
measurement, this enables standard-siren cosmology: a Hubble diagram can be
constructed to directly probe the Universe's expansion. This can be done in the
absence of electromagnetic measurements as features in the mass distribution of
GW sources provide self-calibrating redshift measurements without the need for
a definite or probabilistic host galaxy association. This technique has thus
far only been applied with simple parametric representations of the mass
distribution. However, the use of an inaccurate representation leads to biases
in the cosmological inference, an acute problem given the current uncertainties
in true source population. Furthermore, it is commonly presumed that the form
of the mass distribution must be known a priori to obtain unbiased measurements
of cosmological parameters in this fashion. Here, we demonstrate that spectral
sirens can accurately infer cosmological parameters without such prior
assumptions. We apply a flexible, non-parametric model for the mass
distribution of compact binaries to a simulated catalog of 1,000
gravitational-wave events, consistent with expectations for the next LVK
observing run. We find that, despite our model's flexibility, both the source
mass model and cosmological parameters are correctly reconstructed. We predict
a $5.8\%$ measurement of $H_0$, keeping all other cosmological parameters
fixed, and a $6.4\%$ measurement of $H(z=0.9)$ when fitting for multiple
cosmological parameters ($1\sigma$ uncertainties). This
astrophysically-agnostic spectral siren technique will be essential to arrive
at precise and unbiased cosmological constraints from GW source populations.
