  Longitudinal phase space (LPS) provides a critical information about electron
beam dynamics for various scientific applications. For example, it can give
insight into the high-brightness X-ray radiation from a free electron laser.
Existing diagnostics are invasive, and often times cannot operate at the
required resolution. In this work we present a machine learning-based Virtual
Diagnostic (VD) tool to accurately predict the LPS for every shot using
spectral information collected non-destructively from the radiation of
relativistic electron beam. We demonstrate the tool's accuracy for three
different case studies with experimental or simulated data. For each case, we
introduce a method to increase the confidence in the VD tool. We anticipate
that spectral VD would improve the setup and understanding of experimental
configurations at DOE's user facilities as well as data sorting and analysis.
The spectral VD can provide confident knowledge of the longitudinal bunch
properties at the next generation of high-repetition rate linear accelerators
while reducing the load on data storage, readout and streaming requirements.
