  In order for optical cavities to enable strong light-matter interactions for
quantum metrology, networking, and scalability in quantum computing systems,
their mirrors must have minimal losses. However, high-finesse dielectric cavity
mirrors can degrade in ultra-high vacuum (UHV), increasing the challenges of
upgrading to cavity-coupled quantum systems. We observe the optical degradation
of high-finesse dielectric optical cavity mirrors after high-temperature UHV
bake in the form of a substantial increase in surface roughness. We provide an
explanation of the degradation through atomic force microscopy (AFM), X-ray
fluorescence (XRF), selective wet etching, and optical measurements. We find
the degradation is explained by oxygen reduction in Ta$_2$O$_5$ followed by
growth of tantalum sub-oxide defects with height to width aspect ratios near
ten. We discuss the dependence of mirror loss on surface roughness and finally
give recommendations to avoid degradation to allow for quick adoption of
cavity-coupled systems.
