  We present an unsupervised outlier detection method for galaxy spectra based
on the spectrum autoencoder architecture spender, which reliably captures
spectral features and provides highly realistic reconstructions for SDSS galaxy
spectra. We interpret the sample density in the autoencoder latent space as a
probability distribution, and identify outliers as low-probability objects with
a normalizing flow. However, we found that the latent-space position is not, as
expected from the architecture, redshift invariant, which introduces
stochasticity into the latent space and the outlier detection method. We solve
this problem by adding two novel loss terms during training, which explicitly
link latent-space distances to data-space distances, preserving locality in the
autoencoding process. Minimizing the additional losses leads to a
redshift-invariant, non-degenerate latent space distribution with clear
separations between common and anomalous data. We inspect the spectra with the
lowest probability and find them to include blends with foreground stars,
extremely reddened galaxies, galaxy pairs and triples, and stars that are
misclassified as galaxies. We release the newly trained spender model and the
latent-space probability for the entire SDSS-I galaxy sample to aid further
investigations.
