  Nitrogen-vacancy (NV) centers in diamond are reliable single-photon emitters,
with applications in quantum technologies and metrology. Two charge states are
known for NV centers: NV$^0$ and NV$^-$, with the latter being mostly studied
due to its long electron spin coherence time. Therefore, control over the
charge state of the NV centers is essential. However, an understanding of the
dynamics between the different states still remains challenging. Here,
conversion from NV$^-$ to NV$^0$ due to electron-induced carrier generation is
shown. Ultrafast pump-probe cathodoluminescence spectroscopy is presented for
the first time, with electron pulses as pump, and laser pulses as probe, to
prepare and read out the NV states. The experimental data is explained with a
model considering carrier dynamics (0.8 ns), NV$^0$ spontaneous emission (20
ns) and NV$^0$ $\rightarrow$NV$^-$ back transfer (500 ms). The results provide
new insights into the NV$^-$$\leftrightarrow$NV$^0$ conversion dynamics, and
into the use of pump-probe cathodoluminescence as a nanoscale NV
characterization tool.
