  We demonstrate hybridization of optical cavity photons with atomic Rydberg
excitations using electromagnetically induced transparency (EIT). The resulting
dark state Rydberg polaritons exhibit a compressed frequency spectrum and
enhanced lifetime indicating strong light-matter mixing. We study the coherence
properties of cavity Rydberg polaritons and identify the generalized EIT
linewidth for optical cavities. Strong collective coupling suppresses polariton
losses due to inhomogeneous broadening, which we demonstrate by using different
Rydberg levels with a range of polarizabilities. Our results point the way
towards using cavity Rydberg polaritons as a platform for creating photonic
quantum materials.
