  Current large-scale astrophysical experiments produce unprecedented amounts
of rich and diverse data. This creates a growing need for fast and flexible
automated data inspection methods. Deep learning algorithms can capture and
pick up subtle variations in rich data sets and are fast to apply once trained.
Here, we study the applicability of an unsupervised and probabilistic deep
learning framework, the Probabilistic Autoencoder (PAE), to the detection of
peculiar objects in galaxy spectra from the SDSS survey. Different to
supervised algorithms, this algorithm is not trained to detect a specific
feature or type of anomaly, instead it learns the complex and diverse
distribution of galaxy spectra from training data and identifies outliers with
respect to the learned distribution. We find that the algorithm assigns
consistently lower probabilities (higher anomaly score) to spectra that exhibit
unusual features. For example, the majority of outliers among quiescent
galaxies are E+A galaxies, whose spectra combine features from old and young
stellar population. Other identified outliers include LINERs, supernovae and
overlapping objects. Conditional modeling further allows us to incorporate
additional information. Namely, we evaluate the probability of an object being
anomalous given a certain spectral class, but other information such as metrics
of data quality or estimated redshift could be incorporated as well. We make
our code publicly available at https://github.com/VMBoehm/Spectra_PAE
