  The $^1E-^1A_1$ singlet manifold of the negatively charged nitrogen vacancy
$(NV^-)$ center in diamond plays a central role in the quantum information and
quantum sensing applications of the $NV^-$ center. However, the energy of this
manifold within the diamond bandgap and with respect to the $^3A_2-^3E$ triplet
manifold has not been measured directly. Using field-quenching effects on
photoluminescence (PL) spectra, we report on the energy gap between the
$^1E-^1A_1$ singlet manifold and the $^3A_2$ and $^3E$ ground and excited
triplet states of the $NV^-$ as a function of excitation wavelength and power,
temperature, and applied magnetic field in a heavily nitrogen-doped sample.
Increased PL and decreased zero-phonon line width from the $NV^0$ were observed
in the presence of an applied magnetic field, indicating ionization from the
long-lived $^1E$ singlet state. A temperature-dependent ionization threshold
between 532 nm and 550 nm was found, locating the singlet states within the
diamond band gap.
